System

Abstract

To provide a system.SOLUTION: A system comprises: means for acquiring participant characteristics including areas of expertise and positiveness of participants of a meeting; means for providing speaking opportunities to the participants on the basis of the meeting topics and participant characteristics; means for analyzing the spoken content after the speaking opportunities are provided, and feeding back positive reactions to the speaker; means for detecting silence during the meeting and providing supplementary information related to the topic to the participants when the silence is detected; and means for analyzing emotional states of the participants and responding according to the emotional states.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In meetings, passive participants often do not have the opportunity to speak up, and their opinions are not reflected. There are also problems such as reluctance to speak up and missing the opportunity to speak up. [Means for solving the problem]

[0005] This invention inputs the characteristics of participants, such as their areas of expertise and proactiveness, and provides a means to initiate conversations based on the number of times participants speak and their characteristics, a means to have a positive reaction after starting a conversation, and a means to provide supplementary information related to the topic during silences. This makes it possible to create an environment where even passive participants feel comfortable speaking up. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The present invention is comprised of a data collection module for grasping the characteristics of conference participants, a speech allocation module for providing speaking opportunities, a reaction generation module for generating positive reactions after speaking, and an information provision module for providing supplementary information during silences.

[0029] "Example 2"

[0030] The data collection module analyzes participants' past comments and behavioral patterns to extract characteristics such as their areas of expertise and proactiveness. For example, it is possible to extract participant characteristics by analyzing past meeting minutes and participants' social media posts.

[0031] "Example 3"

[0032] The speech allocation module distributes speaking opportunities equally based on the extracted participant characteristics and the meeting agenda. For example, when a topic on which participant A excels comes up, the opportunity to speak is provided to A. Similarly, participant B, who speaks less frequently, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up.

[0033] "Example 4"

[0034] The reaction generation module generates positive reactions and provides feedback to participants after they speak. For example, it generates positive comments such as, "That's a fresh perspective." This helps participants feel less reluctant to speak up.

[0035] "Example 5"

[0036] The information module provides supplementary information related to the agenda when there is silence in the meeting. For example, if the agenda is "new product development," it can provide information on market trends and competitors' activities. This gives participants an opportunity to speak up from a new perspective.

[0037] The processing flow of each embodiment will be described below.

[0038] "Example 1"

[0039] Step 1: The data collection module runs and analyzes participants' past comments and behavioral patterns. This analysis extracts characteristics such as participants' areas of expertise and proactiveness.

[0040] Step 2: The speech allocation module operates and allocates speaking opportunities equally based on the extracted participant characteristics and the meeting agenda.

[0041] Step 3: The reaction generation module operates to generate positive reactions after a participant speaks and provide feedback to the speaker.

[0042] Step 4: The information provision module operates to provide supplementary information related to the agenda when the meeting is silent.

[0043] "Example 2"

[0044] Step 1: The data collection module runs and analyzes participants' past meeting minutes, their social media posts, etc. This analysis extracts the participants' characteristics.

[0045] Step 2: The speech allocation module operates, and when a topic that Participant A is good at comes up, it provides Participant A with an opportunity to speak. Also, Participant B, who has not spoken often, is provided with an opportunity to speak when a topic related to Participant B's area of ​​expertise comes up.

[0046] Step 3: The reaction generation module operates and generates positive comments such as "That's a fresh perspective" and provides feedback to the participants.

[0047] Step 4: The information provision module operates and, if the topic is "new product development," provides information such as market trends and competitors' activities.

[0048] Example 1

[0049] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0050] Conventional conference systems have difficulty fully understanding the characteristics of participants and providing appropriate opportunities to speak. Furthermore, they lack positive reactions after speaking and supplementary information during silences, making it difficult to facilitate smooth progress in meetings or encourage active participation. This has led to problems such as reduced meeting efficiency and productivity.

[0051] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0052] In this invention, the server includes a means for collecting participant profile information and storing it in a database, a means for generating positive reactions using a generative AI model, and a means for searching for and providing related supplemental information using the generative AI model. This makes it possible to understand the characteristics of participants and provide appropriate opportunities to speak. In addition, by providing positive reactions after speaking and supplemental information during silences, the meeting can proceed smoothly and encourage active participation by participants.

[0053] "Participants' specialties, proactiveness, and other characteristics" refers to the characteristics of each person participating in the meeting, such as their expertise and skills, frequency of speaking, and motivation.

[0054] "Number of speeches" refers to the number of times each participant spoke during the conference.

[0055] A "speaking device" refers to a method or system for providing a specific participant with an opportunity to speak during a meeting.

[0056] "Means for positive reactions" refers to methods or systems for showing positive reactions or evaluations after participants make comments.

[0057] "Means for providing supplementary information related to the topic during silence" refers to a method or system for providing additional information related to the topic when silence occurs during a meeting.

[0058] "Profile Information" refers to personal information about you, such as your name, job title, and area of ​​expertise.

[0059] A "database" refers to an information management system that systematically stores collected information and enables it to be searched and used as needed.

[0060] A "generative AI model" refers to a model that has been trained using artificial intelligence techniques to perform a specific task.

[0061] "Supplementary information" refers to additional data or knowledge relevant to the meeting topic.

[0062] "Means for searching and providing" refers to a method or system for searching the Internet or a database for necessary information and providing it to the user.

[0063] MODE FOR CARRYING OUT THE INVENTION

[0064] This invention is a system that understands the characteristics of conference participants, provides appropriate opportunities to speak, generates positive reactions after speaking, and provides supplementary information during silences. This system works in cooperation with three parties: a server, a terminal, and a user.

[0065] Data Collection Module

[0066] 1. A user logs in to a terminal to participate in a conference.

[0067] 2. The device displays a screen for entering the user's profile information.

[0068] 3. The user enters profile information such as name, job title, and area of ​​expertise.

[0069] 4. The terminal sends the entered information to the server.

[0070] 5. The server stores the received profile information in a database.

[0071] Examples:

[0072] The user enters their name and job title into a form displayed on the terminal screen.

[0073] The terminal transmits the input information to the server in real time.

[0074] The server stores the data in the database in the format "User A: Name = Yamada Taro, Position = Manager, Area of ​​Expertise = Marketing."

[0075] Message sorting module

[0076] 1. The server monitors the progress of the conference and schedules speaking opportunities for each participant.

[0077] 2. The server selects appropriate participants for a particular topic based on the collected data.

[0078] 3. The terminal receives instructions from the server and notifies the selected participants when it is time to speak.

[0079] 4. The user receives the notification and makes a comment.

[0080] Examples:

[0081] The server executes logic such as "encourage Yamada Taro, who has expertise in marketing, to speak on agenda item A."

[0082] The device displays the message "Please speak next, Yamada Taro."

[0083] The user confirms the notification and starts speaking.

[0084] Reaction Generation Module

[0085] 1. The server analyzes the user's comments in real time.

[0086] 2. The server inputs a prompt sentence into the generative AI model to generate a positive reaction.

[0087] 3. The device displays the generated reaction to the user.

[0088] 4. The user checks the displayed reaction.

[0089] Examples:

[0090] The server inputs a prompt statement into the generative AI model: "Analyze what User A said and generate a positive reaction."

[0091] The generative AI model generates the reaction, "That's a great idea, Yamada!"

[0092] The device displays this reaction on the screen.

[0093] The user sees the message, "Great idea, Yamada!"

[0094] Information Module

[0095] 1. The server searches for relevant supplemental information during silences in the meeting.

[0096] 2. The server collects relevant data and selects the appropriate information.

[0097] 3. The terminal displays the supplementary information sent from the server to the user.

[0098] 4. The user checks the displayed supplementary information.

[0099] Examples:

[0100] The server inputs the prompt statement "Find the latest news articles related to the current topic" into the generative AI model.

[0101] Generative AI models search for relevant news articles and statistical data and generate information such as, "Here's an article about the latest marketing trends."

[0102] The terminal displays this information on the screen.

[0103] The user sees a message that reads, "Here's an article about the latest marketing trends."

[0104] Prompt Sentence Examples

[0105] "Analyze what User A said and generate a positive reaction."

[0106] "Find the latest news articles related to your current topic."

[0107] The above is an embodiment of the present invention.

[0108] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0109] Step 1:

[0110] A user logs in to a terminal to participate in a conference.

[0111] Input: User login information (user ID, password)

[0112] Output: Login success or failure message

[0113] Specific operation: The user enters their user ID and password into the login screen on the device and clicks the login button. The device sends the entered information to the server, which then performs authentication. If authentication is successful, a login success message is displayed.

[0114] Step 2:

[0115] The terminal displays a screen for inputting user profile information.

[0116] Input: None

[0117] Output: Profile information input screen

[0118] Specific operation: After successfully logging in, the terminal will display a form for the user to enter profile information. The form includes input fields such as name, job title, and area of ​​expertise.

[0119] Step 3:

[0120] Users enter profile information such as name, job title, and area of ​​expertise.

[0121] Input: Profile information such as name, job title, area of ​​expertise, etc.

[0122] Output: Entered profile information

[0123] Specific operation: The user enters information such as name, job title, and area of ​​expertise into the form on the terminal and clicks the submit button.

[0124] Step 4:

[0125] The terminal transmits the input information to the server.

[0126] Input: Profile information entered by the user

[0127] Output: Result sent to the server

[0128] Specific operation: The device sends the profile information entered by the user to the server, which then verifies the received information and stores it in a database.

[0129] Step 5:

[0130] The server stores the received profile information in a database.

[0131] Input: Profile information sent from the device

[0132] Output: Save results to database

[0133] Specific operation: The server saves the received profile information in the database. If the save is successful, it generates a save success message.

[0134] Step 6:

[0135] The server monitors the progress of the conference and schedules speaking opportunities for each participant.

[0136] Input: Meeting progress, participant profile information

[0137] Output: Speaking Opportunity Plan

[0138] Specific operation: The server plans appropriate speaking opportunities for each participant based on the progress of the meeting and the participant's profile information. The plan includes who will speak on which topic.

[0139] Step 7:

[0140] The server uses the collected data to select appropriate participants for a particular topic.

[0141] Input: Participant profile information, meeting agenda

[0142] Output: Selected participants

[0143] Specific operation: The server analyzes participants' areas of expertise and past comments and selects the most appropriate participants for a particular topic.

[0144] Step 8:

[0145] The terminal receives instructions from the server and notifies the selected participants of the timing to speak.

[0146] Input: Instructions from the server (selected participants, timing to speak)

[0147] Output: Notification of speaking timing

[0148] Specific operation: The terminal receives instructions from the server and displays the message "Please speak next" to the selected participant.

[0149] Step 9:

[0150] The user receives the notification and makes a comment.

[0151] Input: Notification of timing to speak

[0152] Output: What the user said

[0153] Specific operation: The user checks the notification displayed on the device and speaks at the specified timing.

[0154] Step 10:

[0155] The server analyzes the content of the user's comments in real time.

[0156] Input: What the user says

[0157] Output: Analysis results

[0158] Specific operation: The server analyzes the text of the user's comments and extracts important keywords and context.

[0159] Step 11:

[0160] The server inputs prompt sentences into the generative AI model to generate positive reactions.

[0161] Input: Analysis results, prompt text

[0162] Output: Positive reaction

[0163] Specific operation: The server inputs the prompt "Analyze what User A said and generate a positive reaction" into the generative AI model. The generative AI model generates the reaction "That's a great idea, Yamada-san!"

[0164] Step 12:

[0165] The terminal displays the generated reaction to the user.

[0166] Input: Positive reaction

[0167] Output: Show reactions

[0168] Specific behavior: The device displays the generated positive reaction on the screen. The user sees the message, "That's a great idea, Yamada-san!"

[0169] Step 13:

[0170] The server searches for relevant supplemental information during periods of silence in the conference.

[0171] Input: Meeting agenda, silence occurrence

[0172] Output: Supplementary information

[0173] How it works: The server inputs a prompt to the generative AI model: "Find the latest news articles related to the current topic." The generative AI model searches for relevant news articles and statistical data and generates information such as, "Here's an article about the latest marketing trends."

[0174] Step 14:

[0175] The terminal displays the supplementary information sent from the server to the user.

[0176] Input: Supplementary information

[0177] Output: Display supplementary information

[0178] Specific operation: The device displays the supplementary information sent from the server on the screen. The user sees the message "Click here for an article about the latest marketing trends."

[0179] (Application example 1)

[0180] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0181] Conventional conference and communication systems have difficulty fully understanding the characteristics of participants and providing opportunities to speak at appropriate times. Furthermore, they often lack sufficient support for reactions after a comment or supplementary information during silences, hindering smooth communication. Furthermore, in the case of communication between workers and robots in factories, there is a lack of instructions and feedback that take into account the characteristics of the workers, resulting in a decline in work efficiency.

[0182] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0183] In this invention, the server includes means for inputting characteristics of participants such as their areas of expertise and proactiveness, and for initiating conversations based on the number of times participants speak and their characteristics, means for generating positive reactions after initiating conversations, means for providing supplementary information related to the topic during silences, means for collecting characteristic data of workers, means for issuing work instructions at appropriate times, means for generating positive reactions to comments made by workers, and means for providing supplementary information when work is stalled. This enables smooth communication and efficient work instructions that take into account the characteristics of participants and workers.

[0184] "Participant" refers to an individual or group participating in a meeting or task.

[0185] "Area of ​​expertise" refers to a participant's area of ​​particular proficiency or expertise.

[0186] "Proactivity" refers to the attitude and character of participants to act spontaneously and actively express their opinions and ideas.

[0187] "Number of remarks" refers to the number of times a participant spoke during a meeting or task.

[0188] "Methods of starting a conversation" refers to methods and techniques for providing participants with an opportunity to speak and starting a topic at an appropriate time.

[0189] "Positive reactions" refer to having a positive reaction to a participant's words or actions.

[0190] "Silent times" refer to periods when no one is speaking during a meeting or during work.

[0191] "Supplementary information" refers to additional information or explanation related to a topic or task.

[0192] "Worker" refers to an individual who actually performs work in a factory or work site.

[0193] "Characteristic data" refers to data that indicates a worker's characteristics, such as their areas of expertise, experience, and behavioral patterns.

[0194] "Work instructions" refers to giving specific work content and procedures to a worker.

[0195] "Means for generating" refers to methods or technologies for automatically producing specific reactions or information.

[0196] A system for implementing this invention includes means for inputting characteristics such as areas of expertise and proactiveness of participants, and for initiating conversation based on the number of times participants speak and their characteristics, means for generating positive reactions after initiating conversation, means for providing supplementary information related to the topic during periods of silence, means for collecting data on worker characteristics, means for issuing work instructions at appropriate times, means for generating positive reactions to comments made by workers, and means for providing supplementary information when work is stalled.

[0197] The server runs a Python program to collect and analyze data on the characteristics of participants and workers. Specifically, it analyzes participants' past comments and behavioral patterns to extract their areas of expertise and proactiveness. This allows it to allocate conversations in a way that evens out the number of times each participant speaks.

[0198] The server also uses a generative AI model (e.g., GPT-3 (registered trademark)) to generate positive reactions to participants' comments. For example, if a participant asks, "What's the next step?", the server inputs the following prompt sentence into the generative AI model:

[0199] The participant asked, "What's the next step?" Provide a positive reaction and supporting information.

[0200] An example of the output from a generative AI model is the following response:

[0201] Great question! The next step is to attach part A to part B. The key to this step is to check the orientation of part A.

[0202] Additionally, the server provides topic-related supplemental information during silences. For example, if no one is speaking during a meeting, the server uses a generative AI model to provide supplemental information such as:

[0203] The next step in this process is to make sure that part A is oriented correctly when attaching it to part B.

[0204] In this way, the server can realize smooth communication and efficient work instructions that take into account the characteristics of participants and workers. Potential hardware to be used includes robots in factories and smart glasses worn by workers. The software uses Python programs and generative AI models (e.g., GPT-3).

[0205] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0206] Step 1:

[0207] The server collects characteristic data of participants and workers. Specifically, it analyzes participants' past comments and behavioral patterns to extract their areas of expertise and proactiveness. It uses participants' past comment records and behavior logs as input and generates data on participants' areas of expertise and proactiveness as output.

[0208] Step 2:

[0209] The server uses the collected characteristic data to initiate a conversation based on the number of times participants have spoken and their characteristics. Specifically, it initiates a topic to even out the number of times each participant has spoken. As input, it uses the participant's characteristic data and the current number of times they have spoken, and as output, it determines the participant who should speak next.

[0210] Step 3:

[0211] The server generates a positive reaction after initiating the conversation. Specifically, it uses a generative AI model (e.g., GPT-3) to generate a positive reaction to the participant's comments. It uses the participant's comments as input and generates a positive reaction as output.

[0212] Step 4:

[0213] The server provides topic-related supplemental information during silences. Specifically, it uses a generative AI model to generate supplemental information related to meetings and tasks. It uses the current topic and task content as input and generates supplemental information as output.

[0214] Step 5:

[0215] The server collects the characteristic data of the workers and issues work instructions at the appropriate time. Specifically, it generates optimal work instructions based on the worker's expertise and experience. The server uses the worker's characteristic data and current work situation as input and generates work instructions as output.

[0216] Step 6:

[0217] The server generates a positive reaction to the worker's utterance. Specifically, it uses a generative AI model to generate a positive reaction to the worker's utterance. It uses the worker's utterance as input and generates a positive reaction as output.

[0218] Step 7:

[0219] The server provides supplementary information when a task stalls. Specifically, it uses a generative AI model to generate supplementary information related to the task. It uses the current task status as input and generates the supplementary information as output.

[0220] Example 2

[0221] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0222] Conventional conference systems have the problem of reducing meeting efficiency because they initiate conversations without considering participants' areas of expertise or proactiveness. Furthermore, there is a lack of means to analyze participants' past comments and behavioral patterns to extract their characteristics, making it difficult to provide appropriate topics or positive reactions. Furthermore, there are also insufficient means to provide relevant supplementary information during silences.

[0223] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0224] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, means for initiating a conversation based on the number of times participants have spoken and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for collecting participants' past speech content and behavioral patterns, means for preprocessing the collected data, means for extracting characteristics from the preprocessed data, means for evaluating the extracted characteristics, and means for outputting the evaluation results. This enables efficient conference management that takes into account the characteristics of participants.

[0225] "Participants' specialties, proactiveness, and other characteristics" are characteristics that indicate expertise and skills in specific fields, as well as proactiveness in meetings and discussions, extracted from participants' past statements and behavioral patterns.

[0226] "Methods of starting a conversation based on the number of times participants have spoken and their characteristics" refers to a method of providing topics at appropriate times and encouraging participants to speak based on the number of times participants have spoken and their characteristics.

[0227] "Means for a positive reaction" are ways to positively evaluate a participant's comments after they have been made and to increase their motivation.

[0228] "Means for providing additional information related to the topic during silence" refers to methods for providing additional information related to the topic and stimulating conversation when silence occurs during a meeting.

[0229] "Means of collecting participants' past statements and behavioral patterns" refers to methods for collecting data on participants' past statements and behavior, such as meeting minutes and social media posts.

[0230] The "means for preprocessing collected data" refers to a method for performing preprocessing such as removing unnecessary information and tokenizing text to make the collected data easier to analyze.

[0231] The "means for extracting characteristics from preprocessed data" is a method for extracting characteristics such as participants' areas of expertise and proactiveness based on the preprocessed data.

[0232] The "means for evaluating extracted characteristics" is a method for evaluating participants' areas of expertise and proactiveness based on the extracted characteristics, and for using this information to help manage the meeting.

[0233] "Means for outputting evaluation results" refers to a method for outputting the results of the evaluated characteristics in JSON format or the like and sending them to other systems or terminals.

[0234] The present invention is a system for efficiently managing a conference based on the characteristics of participants, such as their areas of expertise and their proactiveness. A specific embodiment of this system will be described below.

[0235] 1. Generating the system program

[0236] The server generates a system program that includes a data collection module. This program analyzes participants' past comments and behavioral patterns, and extracts their characteristics, such as their areas of expertise and proactiveness.

[0237] 2. Explanation of program processing

[0238] The server processes and calculates data using the following hardware and software:

[0239] Hardware and software used

[0240] Hardware: Servers, terminals

[0241] Software: Python, NLTK, SpaCy, scikit-learn, TENSORFLOW (registered trademark), Twitter API, Google (registered trademark) Drive API

[0242] Data collection

[0243] The server retrieves participants' social media posts using the Twitter API, and the meeting minutes using the Google Drive API.

[0244] Data Preprocessing

[0245] The server uses the NLTK library to clean and tokenize the text data, for example splitting the text "I'm good at project management" into "project," "management," and "good at."

[0246] Feature Extraction

[0247] The server uses TF-IDF (Term Frequency-Inverse Document Frequency) to extract important keywords such as "project management" and "leadership." For example, if the TF-IDF score of "project management" is high, the server determines that the word is important.

[0248] Characterization

[0249] The server uses a random forest model from scikit-learn to evaluate participants' areas of expertise and proactiveness. For example, if a participant has a high score in "project management," the server evaluates the participant as being good at project management.

[0250] Output of results

[0251] The server outputs the evaluation results in JSON format and sends them to the terminal. The terminal displays the evaluation results to the user. For example, it displays the result that "Participant A is good at project management and is proactive."

[0252] 3. Examples of concrete examples and prompts

[0253] For example, if keywords such as "project management" and "leadership" appear frequently in the minutes of a meeting, the server will determine that the participant is skilled in project management. Similarly, if phrases such as "active exchange of opinions" and "proposing new ideas" appear frequently in social media posts, the participant will be evaluated as proactive.

[0254] Example prompts to input to the generative AI model:

[0255] "Evaluate participants' areas of expertise and proactiveness based on past meeting minutes and social media posts."

[0256] In this way, the server performs a series of processes from data collection to output of results, realizing efficient conference management that takes into account the characteristics of the participants.

[0257] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0258] Step 1: Data collection

[0259] The server collects participants' past comments and behavioral patterns. Specifically, it uses the Twitter API to obtain participants' social media posts and the Google Drive API to obtain meeting minutes. The input is raw data obtained through the API, and the output is the collected text data.

[0260] Step 2: Data Preprocessing

[0261] The server preprocesses the collected data. Specifically, it uses the NLTK library to clean and tokenize the text data. For example, it splits the text "I'm good at project management" into "project," "management," and "good at." The input is the collected text data, and the output is the preprocessed, clean text data.

[0262] Step 3: Feature extraction

[0263] The server extracts features from the preprocessed data. Specifically, it uses TF-IDF (Term Frequency-Inverse Document Frequency) to extract important keywords and phrases. For example, if the TF-IDF score of "project management" is high, it determines that the word is important. The input is the preprocessed clean text data, and the output is the extracted keywords and phrases.

[0264] Step 4: Characterization

[0265] The server evaluates participants' characteristics, such as their areas of expertise and proactiveness, based on the extracted keywords and phrases. Specifically, it uses a random forest model from scikit-learn to learn from past data and predict characteristics for new data. The input is the extracted keywords and phrases, and the output is a score for the evaluated characteristics.

[0266] Step 5: Output the results

[0267] The server outputs the evaluation results in JSON format and sends them to the terminal. The terminal displays the evaluation results to the user. For example, it displays the result "Participant A is good at project management and is proactive." The input is the score of the evaluated characteristics, and the output is the evaluation result displayed to the user.

[0268] (Application example 2)

[0269] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0270] Conventional systems have the problem of reducing meeting and work efficiency because they initiate conversations without considering the areas of expertise or enthusiasm of the participants. Also, when allocating work within a factory, they assign tasks without considering the areas of expertise or work efficiency of the workers, which reduces production efficiency. To solve these problems, a system is needed that accurately grasps the characteristics of participants and workers, and based on that, can initiate conversations and allocate work optimally.

[0271] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0272] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, and for initiating conversations based on the number of comments and participant characteristics, means for providing positive reactions after initiating conversations, means for providing supplementary information related to the topic during silences, means for analyzing the past work history and behavioral patterns of workers to identify their areas of expertise and work efficiency, means for automatically assigning work based on their areas of expertise, and means for providing appropriate feedback when a decline in work efficiency is observed. This allows for an accurate understanding of the characteristics of participants and workers, and enables optimal conversation initiation and task assignment based on this information.

[0273] "Participant" refers to an individual who participates in a meeting or task.

[0274] "Area of ​​expertise" refers to an area in which a participant or worker has particularly outstanding abilities or knowledge.

[0275] "Proactivity" refers to the attitude of participants and workers to act voluntarily and take on a proactive approach.

[0276] "Number of times spoken" refers to how often participants speak in meetings or communication settings.

[0277] "Features" refer to individual characteristics extracted from the behavioral patterns and past statements of participants and workers.

[0278] "Starting a conversation" refers to the act of providing a topic to a specific participant and encouraging them to speak.

[0279] "Positive reactions" refer to the act of showing a positive response to the comments or actions of other participants.

[0280] "Silence" refers to a time when no one is speaking during a meeting or communication session.

[0281] "Supplemental information" refers to additional information or explanation related to the topic.

[0282] "Work history" refers to a record of work that a worker has done in the past.

[0283] "Behavioral patterns" refer to the past behavioral tendencies and habits of participants or workers.

[0284] "Work efficiency" refers to an index that indicates how efficiently a worker performs work within a certain period of time.

[0285] "Work assignment" refers to the act of assigning a specific task to a specific worker.

[0286] "Feedback" refers to evaluations and advice provided to workers regarding their behavior and performance.

[0287] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The server includes a data collection module that analyzes the past comments and behavioral patterns of participants and workers to extract characteristics such as areas of expertise and proactiveness. The terminal is a tablet or smartphone used by the worker, and provides an interface for receiving feedback and work assignments from the server. The user is an individual who participates in meetings and work, and acts based on the information provided by the system.

[0288] The server processes and calculates data using software such as Python, Pandas, and Scikit-learn. Specifically, it vectorizes text data using TfidfVectorizer and analyzes the data using KMeans clustering. This allows for an accurate understanding of the characteristics of participants and workers, and allows for optimal topic assignment and task allocation based on that information.

[0289] For example, the server may analyze the past work history of worker A and determine that he is good at welding. It may also analyze the social media posts of worker B and determine that he has a tendency to proactively try new tasks. Based on this, the server may assign welding work to worker A and new work to worker B. Furthermore, if a decline in work efficiency is observed, the server may provide appropriate feedback.

[0290] Examples of prompts using generative AI models include:

[0291] "Analyze worker A's past work history and identify the tasks he excels at. Also, analyze worker B's social media posts and evaluate his proactiveness."

[0292] In this way, a system can be realized that accurately grasps the characteristics of participants and workers and, based on that, optimally initiates topics and assigns tasks.

[0293] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0294] Step 1:

[0295] The server collects the worker's past work history data and SNS data.

[0296] Input: Work history data (CSV file), SNS data (CSV file)

[0297] Output: Work history data frame, SNS data frame

[0298] Specific operation: The server reads the work history data and SNS data from the CSV file and converts them into a data frame using Pandas.

[0299] Step 2:

[0300] The server analyzes the work history data and identifies the worker's specialty.

[0301] Input: Work history data frame

[0302] Output: Specialty clusters for each worker

[0303] Specific operation: The server vectorizes the work history data using TfidfVectorizer and identifies each worker's areas of expertise using KMeans clustering.

[0304] Step 3:

[0305] The server analyzes the SNS data and evaluates the proactiveness of the worker.

[0306] Input: SNS data frame

[0307] Output: Proactivity clusters for each worker

[0308] Specific operation: The server vectorizes the SNS data using TfidfVectorizer and evaluates the proactiveness of each worker using KMeans clustering.

[0309] Step 4:

[0310] The server automatically assigns work based on expertise and proactiveness.

[0311] Input: Specialty cluster and proactiveness cluster for each worker

[0312] Output: Work assignment list

[0313] Specific operation: The server executes an algorithm to assign the most suitable tasks to each worker based on cluster information on their areas of expertise and aggressiveness.

[0314] Step 5:

[0315] The server provides appropriate feedback when a decrease in work efficiency is observed.

[0316] Input: Work assignment list, work efficiency data

[0317] Output: Feedback message

[0318] Specific operation: The server monitors work efficiency data, and if efficiency decreases, it generates a feedback message and sends it to the terminal.

[0319] Step 6:

[0320] The terminal receives feedback and work assignments from the server and notifies the user.

[0321] Input: Feedback message, work assignment list

[0322] Output: User notification

[0323] Specific operation: The terminal displays and notifies the user of the feedback messages and work assignment list received from the server.

[0324] Step 7:

[0325] The user acts based on the notification from the terminal and performs the task.

[0326] Input: User notification

[0327] Output: Work performed

[0328] Specific actions: The user checks the notification from the device and performs the instructed task.

[0329] Example 3

[0330] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0331] In conventional conference systems, participants' opportunities to speak tend to be uneven, and speech is often concentrated on certain participants. Furthermore, due to a lack of feedback on speech, participants may feel reluctant to speak up. Furthermore, when there is silence in the meeting, no supplementary information related to the agenda is provided, which can stall the progress of the meeting. To solve these problems, a system is needed that distributes speaking opportunities equally based on the characteristics of participants, provides positive reactions to speech, and provides supplementary information when there is silence in the meeting.

[0332] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0333] In this invention, the server includes means for collecting participant characteristic data and agenda data, means for analyzing the collected data, means for equally allocating speaking opportunities based on the analysis results, means for acquiring participants' speech content in real time, means for analyzing the acquired speech content and generating positive reactions using a generative AI model, means for monitoring the progress of the meeting in real time, means for searching for supplementary information related to the agenda, and means for providing related supplementary information based on the search results. This makes it possible to equally allocate speaking opportunities based on participants' characteristics, provide positive feedback for speech, and provide supplementary information when the meeting becomes silent.

[0334] "Participant characteristic data" refers to information about meeting participants, such as their areas of expertise, their level of enthusiasm, past comments, and behavioral patterns.

[0335] "Agenda data" refers to information about the themes or topics to be covered at a meeting.

[0336] "Means of collection" refers to the methods and techniques used to obtain participant characteristic data and agenda data.

[0337] "Means of analysis" refers to methods and techniques for analyzing collected data and extracting relevance between participant characteristics and agenda items.

[0338] "Means for allocating speaking opportunities equally" refers to methods and techniques for fairly distributing speaking opportunities, taking into account the characteristics of participants and the number of times they speak.

[0339] "Means of obtaining information in real time" refers to methods and technologies for instantly obtaining the content of participants' comments.

[0340] "Generative AI model" refers to a model that uses artificial intelligence to generate text and comments.

[0341] "Means for generating positive reactions" refers to methods and techniques for generating positive feedback in response to participants' comments.

[0342] "Means for monitoring the progress of a meeting in real time" refers to methods and technologies for instantly monitoring the progress of a meeting.

[0343] "Means for retrieving supplemental information" refers to methods and techniques for retrieving additional information related to the topic.

[0344] "Means for providing supplemental information" refers to methods and techniques for providing participants with the retrieved supplemental information.

[0345] This invention is a system that allocates speaking opportunities equally based on the characteristics of meeting participants, provides positive reactions to comments, and provides supplementary information when the meeting falls silent. Specific embodiments of this system are described below.

[0346] Hardware and software used

[0347] The server uses the following hardware and software:

[0348] Hardware: powerful processor, sufficient memory, storage devices, and network interfaces

[0349] Software: Google Cloud Natural Language API, generative AI models (e.g., OpenAI's GPT-4), ElasticSearch

[0350] Data collection and analysis

[0351] The server collects participant characteristic data and agenda data for the meeting. The participant characteristic data includes areas of expertise, proactiveness, past comments, and behavioral patterns. The agenda data includes information about the themes and topics to be covered in the meeting.

[0352] The server uses the Google Cloud Natural Language API to analyze the collected data, thereby extracting the relevance of participants' characteristics to the agenda.

[0353] Allocation of speaking opportunities

[0354] Based on the analysis results, the server runs an algorithm to distribute speaking opportunities equally. For example, when a topic on which participant A excels comes up, the opportunity to speak is provided to A. Similarly, participant B, who has spoken less frequently, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up.

[0355] Examples:

[0356] If participant A is knowledgeable about "marketing," offer A the opportunity to speak when the topic is "marketing strategy for a new product."

[0357] If participant B is knowledgeable about "technological development" but does not speak often, provide B with an opportunity to speak when the topic is "introduction of new technology."

[0358] Example prompt sentence:

[0359] "Participant A at the meeting is knowledgeable about marketing. If the topic is marketing strategies for a new product, give A a chance to speak."

[0360] Reaction Generation

[0361] The server receives the speeches of the participants in real time and converts them into text using, for example, speech recognition technology.

[0362] The server analyzes the acquired comments and generates positive reactions using a generative AI model (e.g., OpenAI's GPT-4). The generated reactions are fed back to the speaker. For example, a comment such as "That's a fresh perspective" is displayed on the speaker's device.

[0363] Examples:

[0364] If participant C makes a comment about "new product design," the server generates a positive comment such as "That's a fresh perspective" and displays it on C's terminal.

[0365] Example prompt sentence:

[0366] "Participant C made a comment about the design of a new product. Please generate a positive response."

[0367] Providing information

[0368] The server monitors the progress of the conference in real time, for example detecting when no one speaks for a certain period of time.

[0369] The server uses Elasticsearch to find additional information related to the topic, for example, market trends and competitor activity related to "new product development."

[0370] The server provides participants with related supplementary information based on the search results, for example by displaying it on the conference screen.

[0371] Examples:

[0372] If the topic is "new product development" and the meeting falls silent, the server will display information such as "current market trends" and "new product trends of competitors" on the meeting screen.

[0373] Example prompt sentence:

[0374] "The topic is new product development and the meeting is silent. Please provide any relevant supporting information."

[0375] The above is a specific embodiment for carrying out the present invention. This system makes it possible to equally allocate speaking opportunities based on the characteristics of participants, provide positive feedback for comments, and provide supplementary information when the meeting falls silent. The flow of the identification process in the third embodiment will be described with reference to FIG. 15.

[0376] Step 1:

[0377] Data collection

[0378] The server collects meeting participant characteristics and agenda data. As input, it retrieves information such as participants' areas of expertise, proactiveness, past comments, and behavioral patterns from a database. As output, it prepares this data as a dataset for analysis. Specifically, the server sends queries to the database to retrieve the required information.

[0379] Step 2:

[0380] Data analysis

[0381] The server uses the Google Cloud Natural Language API to analyze the collected data. As input, it uses the characteristic data and agenda data collected in step 1. As output, it extracts the relevance between participant characteristics and agendas. Specifically, the server calls the API, analyzes the text data, and generates a relevance score.

[0382] Step 3:

[0383] Allocation of speaking opportunities

[0384] Based on the analysis results, the server runs an algorithm to distribute speaking opportunities equally. As input, it uses the relevance scores obtained in step 2 and the number of times each participant has spoken. As output, it determines the allocation of speaking opportunities to each participant. Specifically, the server runs the algorithm to distribute speaking opportunities equally.

[0385] Step 4:

[0386] Acquiring the content of comments

[0387] The server receives the speeches made by participants in real time. As input, it converts the speeches into text using speech recognition technology. As output, it obtains the textual speeches. Specifically, the server calls a speech recognition API and converts the speech data into text data.

[0388] Step 5:

[0389] Reaction Generation

[0390] The server analyzes the acquired utterances and generates a positive reaction using a generative AI model (e.g., OpenAI's GPT-4). As input, it uses the textual utterances obtained in step 4. As output, it obtains the generated positive reaction. Specifically, the server invokes the generative AI model to generate a positive comment for the utterances.

[0391] Step 6:

[0392] Providing Feedback

[0393] The server feeds back the generated reaction to the speaker. As input, it uses the positive reaction generated in step 5. As output, it displays the reaction on the speaker's device. Specifically, the server sends a message to the speaker's device and displays the reaction.

[0394] Step 7:

[0395] Meeting Monitoring

[0396] The server monitors the progress of the conference in real time. As input, it uses the conference audio data and timestamps of comments. As output, it detects whether the conference is silent or not. Specifically, the server analyzes the audio data and detects when no one has spoken for a certain period of time.

[0397] Step 8:

[0398] Information Search

[0399] The server uses Elasticsearch to search for supplementary information related to the agenda. As input, it uses the silence state detected in step 7 and the agenda data. As output, it obtains the relevant supplementary information. Specifically, the server sends a query to Elasticsearch to search for the relevant information.

[0400] Step 9:

[0401] Providing information

[0402] The server provides participants with relevant supplementary information based on the search results. The supplementary information obtained in step 8 is used as input. The output is to display the supplementary information on the conference screen. Specifically, the server sends a message to the conference screen and displays the supplementary information.

[0403] The above are the specific processing steps of the program of this system.

[0404] (Application example 3)

[0405] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0406] In conventional conferencing and communication systems, participants' opportunities to speak tend to be unevenly distributed, with speech often concentrated on certain participants. Furthermore, a lack of feedback after a comment can cause participants to lose motivation to speak up. Furthermore, when meetings or work stalls, a lack of appropriate supplementary information makes it difficult to make efficient decisions. These problems are particularly pronounced in communication between workers and robots in factories, leading to reduced work efficiency.

[0407] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0408] In this invention, the server inputs characteristics of participants such as their areas of expertise and proactiveness, and includes means for initiating a conversation based on the number of times each participant speaks and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for facilitating communication between workers and robots in a factory, means for equally allocating speaking opportunities, means for providing positive feedback after a speech, and means for providing relevant supplementary information when work stalls. This allows all participants to have an equal opportunity to speak, encourages proactive speaking through feedback after a speech, and provides appropriate supplementary information when the meeting or work stalls, enabling efficient decision-making.

[0409] "Participants" refer to people or robots who take part in a meeting or task.

[0410] "Area of ​​expertise" refers to an area in which a participant has particular knowledge and experience and can demonstrate outstanding abilities.

[0411] "Proactivity" refers to the characteristic of participants acting spontaneously and showing a positive attitude toward speaking and working.

[0412] "Number of remarks" refers to the number of times a participant spoke during a meeting or during work.

[0413] "Methods of initiating conversation" refers to methods of providing participants with opportunities to speak at appropriate times, taking into consideration their characteristics and the number of times they have spoken.

[0414] "Positive reactions" refer to providing positive feedback to participants' comments.

[0415] "Silence" refers to a situation in which no one speaks during a meeting or work session and progress is stagnant.

[0416] "Supplementary Information" refers to additional information related to the agenda or work.

[0417] "Factory workers" refers to human workers in factories.

[0418] A "robot" refers to an automated mechanical device that performs work in a factory.

[0419] "Means for facilitating communication" refers to methods for smoothly exchanging information between workers and robots.

[0420] "Means to distribute speaking opportunities equally" refers to methods that provide all participants with a fair chance to speak.

[0421] "Means of providing positive feedback after a comment" refers to a method of providing a positive response after a participant has commented.

[0422] "Means for providing relevant supplementary information when work stalls" refers to a method for providing useful information to resume work when progress is stalled.

[0423] To implement this invention, it is necessary to build a system that facilitates communication between factory workers and robots. This system inputs the characteristics of participants, such as their areas of expertise and proactiveness, and includes a means for initiating conversations based on the number of times a participant speaks and their characteristics, a means for providing positive reactions after initiating a conversation, and a means for providing supplementary information related to the topic during periods of silence.

[0424] System configuration

[0425] The system consists of the following hardware and software:

[0426] Hardware: Robots in factories, tablets and smartphones used by workers

[0427] Software: A communication system implemented in Python

[0428] Program processing

[0429] The server analyzes participants' past comments and behavioral patterns to input characteristics such as their areas of expertise and proactiveness. This allows it to extract the participants' characteristics. Next, it distributes speaking opportunities equally using a method to allocate speaking opportunities based on the number of comments and participant characteristics. For example, if the topic is "new product development," it provides speaking opportunities to workers who are knowledgeable in that field.

[0430] After a comment is made, positive reactions are generated and feedback is provided to the speaker. This reduces participants' reluctance to speak up. Furthermore, when a meeting or task stalls, supplementary information related to the agenda is provided. For example, information on market trends and competitors' activities regarding "new product development" is provided.

[0431] Specific examples

[0432] For example, the following prompts can be fed to a generative AI model:

[0433] Agenda: New product development

[0434] Worker, please speak about your area of ​​expertise, new product development.

[0435] By inputting this prompt into the generative AI model, it is expected that workers will express their opinions regarding the development of new products. After the comments are made, positive feedback such as "That's a fresh perspective" will be automatically generated.

[0436] In this way, the system ensures that all participants have an equal opportunity to speak, and encourages proactive participation through feedback after speaking. It also provides appropriate supplementary information when meetings or work stalls, enabling efficient decision-making.

[0437] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0438] Step 1:

[0439] The server inputs the participants' characteristics, such as their areas of expertise and proactiveness. Specifically, it analyzes the participants' past comments and behavioral patterns to extract their characteristics. This process takes the participants' comment history data as input and uses a text analysis algorithm to identify their areas of expertise and proactiveness. The output is characteristic data for each participant.

[0440] Step 2:

[0441] The server allocates speaking opportunities equally using a method based on the number of times each participant speaks and the characteristics of each participant. Specifically, it receives the current agenda and participant characteristics as input and provides speaking opportunities to appropriate participants. This process matches the agenda with the participants' areas of expertise and selects the most suitable participant. The output is the name of the selected participant and a speaking prompt.

[0442] Step 3:

[0443] The server sends a speech prompt to the selected participants. Specifically, it sends the generated speech prompt to the participant's terminal. In this process, the server receives the speech prompt as input and sends a message to the participant's terminal. As an output, it generates a speech prompt that is displayed on the participant's terminal.

[0444] Step 4:

[0445] A participant makes a statement. Specifically, the participant inputs the content of the statement through their terminal. In this process, the participant's statement is received as input and sent to the server. The participant's statement data is generated as output.

[0446] Step 5:

[0447] The server generates positive reactions after a comment and provides feedback to the commenter. Specifically, it receives the participant's comment data as input and generates positive feedback using a generative AI model. This process analyzes the content of the comment and generates appropriate positive feedback. A positive feedback message is generated as the output.

[0448] Step 6:

[0449] The server sends the generated positive feedback message to the speaker's terminal. Specifically, the server receives the generated feedback message as input and sends the message to the speaker's terminal. As an output, the server generates a feedback message that is displayed on the speaker's terminal.

[0450] Step 7:

[0451] The server provides supplementary information related to the agenda when a meeting or task stalls. Specifically, it receives the current agenda and related database as input, and searches and extracts relevant supplementary information. This process retrieves information related to the agenda from the database and generates appropriate supplementary information. As an output, a supplementary information message is generated.

[0452] Step 8:

[0453] The server transmits the generated supplemental information message to the participant's terminal. Specifically, the server receives the generated supplemental information message as input and transmits the message to the participant's terminal. As an output, the server generates a supplemental information message that is displayed on the participant's terminal.

[0454] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0455] "Example 1"

[0456] One embodiment of the present invention provides a system that inputs characteristics such as participants' areas of expertise and proactiveness, and includes a means for initiating a conversation based on the number of times participants speak and their characteristics, a means for providing positive reactions after initiating a conversation, a means for providing supplementary information related to the topic during silences, and an emotion engine that recognizes participants' emotions. This system encourages participants to speak and invigorates the meeting. Specifically, it selects the timing and topic for initiating a conversation based on information obtained from the content of participants' comments and behavioral patterns. After a participant speaks, it generates a positive reaction to the comment and provides feedback to the participant. Furthermore, when there is silence in the meeting, it provides supplementary information related to the topic, allowing the meeting to proceed smoothly. The emotion engine that recognizes participants' emotions then grasps the participants' emotional state and responds accordingly.

[0457] "Example 2"

[0458] Another embodiment of the present invention provides a system that inputs characteristics of participants, such as their areas of expertise and proactiveness, and includes a means for initiating a conversation based on the number of times participants speak and their characteristics, a means for providing positive reactions after initiating a conversation, a means for providing supplementary information related to the topic during silences, and an emotion engine for recognizing participants' emotions. This system analyzes participants' past meeting minutes and their social media posts to extract their characteristics. Specifically, when a topic on which Participant A excels comes up, it provides Participant A with an opportunity to speak. Similarly, Participant B, who speaks less frequently, is provided with an opportunity to speak when a topic related to Participant B's area of ​​expertise comes up. Furthermore, it generates positive comments such as "That perspective is fresh" and provides feedback to the participants. If the topic is "new product development," it provides information such as market trends and competitors' activities. The emotion engine that recognizes participants' emotions then grasps their emotional state and responds accordingly.

[0459] "Example 3"

[0460] One embodiment of the present invention provides a system that inputs characteristics such as participants' areas of expertise and proactiveness, and includes a means for initiating a conversation based on the number of times participants speak and their characteristics, a means for providing positive reactions after initiating a conversation, a means for providing supplementary information related to the topic during silences, and an emotion engine that recognizes participants' emotions. This system encourages participants to speak and invigorates the meeting. Specifically, it selects the timing and topic for initiating a conversation based on information obtained from the content of participants' comments and behavioral patterns. After a participant speaks, it generates a positive reaction to the comment and provides feedback to the participant. Furthermore, when there is silence in the meeting, it provides supplementary information related to the topic, allowing the meeting to proceed smoothly. The emotion engine that recognizes participants' emotions then grasps the participants' emotional state and responds accordingly.

[0461] The processing flow of each embodiment will be described below.

[0462] "Example 1"

[0463] Step 1: Input the participants' characteristics, such as their areas of expertise and proactiveness. This is done by analyzing the participants' past statements and behavioral patterns.

[0464] Step 2: Start a conversation based on the number of comments and the characteristics of the participants. Specifically, select the timing and topic to start a conversation based on information obtained from the content of the participants' comments and their behavioral patterns.

[0465] Step 3: React positively after the discussion. This is done by generating a positive reaction to the discussion and providing feedback to the participants.

[0466] Step 4: Provide supplementary information related to the topic during silences. Specifically, when there is silence in the meeting, provide supplementary information related to the topic to keep the meeting running smoothly.

[0467] Step 5: Recognize participants' emotions. Use the emotion engine to understand the participants' emotional state and respond accordingly.

[0468] "Example 2"

[0469] Step 1: Input the participants' characteristics, such as their areas of expertise and proactiveness. This is done by analyzing the participants' past meeting minutes and social media posts to extract their traits.

[0470] Step 2: Initiate the discussion based on the number of times participants speak and the characteristics of the participants. Specifically, when a topic that participant A is good at comes up, provide A with an opportunity to speak. Also, provide participant B, who speaks less, with an opportunity to speak when a topic related to B's area of ​​expertise comes up.

[0471] Step 3: After bringing up the topic, have a positive reaction. Specifically, generate a positive comment such as, "That's a fresh perspective," and provide feedback to the participants.

[0472] Step 4: During silences, provide additional information related to the topic. For example, if the topic is "new product development," provide information on market trends and competitors' activities.

[0473] Step 5: Recognize participants' emotions. Use the emotion engine to understand the participants' emotional state and respond accordingly.

[0474] Example 1

[0475] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0476] Conventional conference systems have difficulty fully understanding the characteristics and emotions of participants and providing appropriate opportunities to speak. Furthermore, they lack sufficient feedback after speaking and supplementary information during silences, making it difficult to liven up the meeting. Furthermore, they are unable to recognize participants' emotional states in real time and respond accordingly, which can lead to a decline in participant satisfaction.

[0477] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0478] In this invention, the server includes means for inputting characteristics of participants, such as their areas of expertise and proactiveness, and for initiating a conversation based on the number of times they speak and their characteristics; means for providing positive reactions after initiating a conversation; means for providing supplementary information related to the topic during silences; means for recognizing participants' emotions; means for analyzing collected data; means for providing speaking opportunities based on the analysis results; means for displaying generated feedback; means for collecting related information; and means for displaying emotional states. This makes it possible to grasp participants' characteristics and emotions and provide appropriate speaking opportunities. Furthermore, providing positive feedback after speaking and supplementary information during silences can help to invigorate the meeting. Furthermore, by recognizing participants' emotional states in real time and responding accordingly, participant satisfaction can be improved.

[0479] "Participant" refers to an individual person taking part in a meeting or discussion.

[0480] "Area of ​​expertise" refers to the area of ​​expertise in which the participant has particular knowledge or skills.

[0481] "Proactivity" refers to participants' willingness and attitude to act voluntarily and express their opinions.

[0482] "Number of remarks" refers to the number of times a participant spoke during a conference.

[0483] "Characteristics" refers to individual attributes of participants, such as their behavioral patterns, personality, and expertise.

[0484] "To raise a topic" means to offer a specific participant an opportunity to speak.

[0485] "Positive reactions" refer to providing positive feedback to participants' comments.

[0486] "Silence" refers to a state in which no one speaks during a meeting.

[0487] "Supplementary Information" refers to additional information or data related to the topic of the conference.

[0488] "Recognizing emotions" refers to judging a participant's emotional state from their facial expressions, tone of voice, etc.

[0489] "Analyzing data" refers to analyzing collected information using statistical or machine learning techniques.

[0490] "Providing an opportunity to speak" means giving participants the opportunity to speak.

[0491] "Displaying feedback" refers to presenting the generated feedback in a visible form to the participants.

[0492] "Collecting relevant information" refers to obtaining information related to the topic of the meeting from the Internet or databases.

[0493] "Displaying an emotional state" refers to visually presenting the perceived emotional state to the participant.

[0494] MODE FOR CARRYING OUT THE INVENTION

[0495] The present invention relates to a system for identifying characteristics of conference participants, providing opportunities to speak, generating positive reactions, and providing supplementary information during silences. Specific embodiments of this system will be described below.

[0496] Data Collection Module

[0497] The server uses a data collection module to understand the characteristics of the conference participants. This module inputs characteristics such as participants' areas of expertise and proactiveness. Specifically, it collects participants' profile information and past comment history and stores them in a database. The software used is a database management system (e.g., MySQL (registered trademark)).

[0498] Data analysis

[0499] The server uses machine learning algorithms to analyze the collected data, specifically, scikit-learn to model participants' behavioral patterns, and the terminal provides an interface to display the analysis results.

[0500] Message sorting module

[0501] The server uses a speech allocation module based on the analysis results to provide participants with speaking opportunities. This module analyzes the number of times and characteristics of participants' speech and allocates the opportunity to speak at the appropriate time. The terminal displays instructions for speaking. The user speaks according to the instructions.

[0502] Reaction Generation Module

[0503] The server uses a reaction generation module to generate positive reactions after participants make comments. Specifically, it uses natural language generation (NLG) technology (e.g., GPT-3) to generate appropriate feedback. The terminal displays the generated feedback. The user receives the feedback.

[0504] Information Module

[0505] The server uses an information provision module to provide supplementary information when the meeting is silent. This module searches for and provides information related to the meeting topic. Specifically, it uses web scraping technology (e.g., Beautiful Soup) to collect relevant information from the Internet. The terminal displays the collected information. The user confirms the provided information.

[0506] Emotion Engine

[0507] The server uses an emotion engine to recognize participants' emotions. This engine uses emotion recognition algorithms (e.g., OpenCV and deep learning models) to analyze participants' facial expressions and tone of voice. Specifically, it analyzes data collected through cameras and microphones to understand participants' emotional states. The terminal displays the emotional states. Users can check their own emotional states.

[0508] Examples and prompts

[0509] Specific examples

[0510] For example, if participant A refrains from speaking during a meeting, the server uses the speech distribution module to display on the terminal, "Mr. A, what do you think about this?" If participant A then says, "I agree with this idea," the server uses the reaction generation module to generate positive feedback, such as "That's a great idea!", and displays this on the terminal. If the meeting falls silent, the server uses the information provision module to display on the terminal, "There are recent research results related to this topic."

[0511] Prompt Sentence Examples

[0512] "Describe a system that captures meeting participant characteristics, provides opportunities to speak, generates positive reactions, and provides additional information during silences."

[0513] In this way, the roles of the server, terminal, and user are clarified, and the processing of the system is explained in detail.

[0514] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0515] Step 1:

[0516] The server starts the data collection module and displays a participant information input screen on the terminal. The user inputs their areas of expertise and past comment history. The input data is sent to the server and stored in a database. A specific example is when the user inputs, "I'm good at marketing."

[0517] Step 2:

[0518] The server uses a machine learning algorithm to analyze the collected data. Specifically, it uses scikit-learn to model participants' behavioral patterns. The input is the participants' areas of expertise and speech history stored in a database. The output is a classification of the participants' characteristics. The terminal displays the analysis result, saying, "Person A is good at marketing."

[0519] Step 3:

[0520] The server uses a speech distribution module based on the analysis results to display the message "Mr. A, what do you think about this?" on the terminal. The analysis results and the current progress of the meeting are used as input. As output, a speech instruction is generated. The user follows the instruction and says "I agree with this idea."

[0521] Step 4:

[0522] The server uses a reaction generation module to generate a positive reaction after a participant makes a comment. Specifically, it uses GPT-3 to generate feedback such as "That's a great idea!". The participant's comment is used as input. The output is the generated positive feedback. The device displays this feedback, and the user replies, "Thank you."

[0523] Step 5:

[0524] The server uses an information provision module to provide supplementary information when the meeting is silent. Specifically, it uses Beautiful Soup to collect relevant information from the Internet. The current meeting topic is used as input. The relevant supplementary information is generated as output. The terminal displays the collected information as "There are recent research results related to this topic." The user reviews the provided information and comments, "This is useful."

[0525] Step 6:

[0526] The server uses an emotion engine to recognize participants' emotions. Specifically, it uses OpenCV and deep learning models to analyze data collected through cameras and microphones. The inputs are participants' facial expressions and tone of voice. The output is the participant's recognized emotional state. The device displays the emotional state as "Person A is currently excited." The user checks their own emotional state and recognizes that "Yes, I am excited."

[0527] (Application example 1)

[0528] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0529] Conventional meeting support systems and work support systems have difficulty fully understanding the characteristics of participants and workers and providing comments and work instructions at the appropriate time. Furthermore, feedback after comments and work was insufficient, making it difficult to maintain the motivation of participants and workers. Furthermore, there was a lack of means to provide appropriate supplementary information during periods of silence or when work stalls, which led to problems with reduced meeting and work efficiency.

[0530] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0531] In this invention, the server inputs characteristics of participants such as their areas of expertise and proactiveness, and includes means for initiating conversations based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating conversations, means for providing supplementary information related to the topic during silences, means for understanding the characteristics of workers and issuing work instructions at appropriate times, means for providing positive feedback after work is completed, means for providing supplementary information when work stalls, and means for understanding the emotional state of workers and responding accordingly.This makes it possible to improve the efficiency of meetings and work and maintain the motivation of participants and workers.

[0532] "Participant" refers to a person who takes part in a meeting or task.

[0533] "Area of ​​expertise" refers to the areas or skills in which a participant or worker is particularly skilled.

[0534] "Proactivity" refers to the willingness and attitude of participants and workers to act voluntarily.

[0535] "Number of times of speaking" refers to the frequency with which participants speak in a meeting.

[0536] "Characteristics" refers to the individual qualities and behavioral patterns of participants or workers.

[0537] "To raise a topic" means to offer a specific participant an opportunity to speak.

[0538] "Positive reaction" refers to a positive response to the comments or actions of participants or workers.

[0539] "Silence" refers to a temporary stagnation in a meeting or work.

[0540] "Supplementary Information" refers to additional information provided to facilitate a meeting or task.

[0541] "Worker" refers to a person who performs work in a factory or work site.

[0542] "Work instructions" refers to instructions to workers regarding the work content and procedures to be performed.

[0543] "Feedback" refers to evaluations and comments given to workers regarding their actions and results.

[0544] "Emotional state" refers to the current emotional or psychological state of a participant or worker.

[0545] To implement this invention, it is necessary to build a system in which a server, a terminal, and a user work together. A specific embodiment of this system is shown below.

[0546] System Configuration

[0547] The server runs a program that includes the following modules:

[0548] Data collection module: Collects participants' and workers' areas of expertise, proactiveness, past statements, and behavioral patterns.

[0549] Speech allocation module: Provides speaking opportunities at appropriate times based on collected data.

[0550] Reaction generation module: Generates positive reactions after a comment or task is completed.

[0551] Information module: Provides supplementary information during periods of silence or when work is stalled.

[0552] Emotion Engine: Recognizes the emotional state of participants and workers and responds accordingly.

[0553] Hardware and Software

[0554] Hardware: Factory work support robots, cameras, microphones

[0555] software:

[0556] EmotionEngine: An engine for recognizing employee emotions

[0557] PositiveFeedbackGenerator: A module that generates positive feedback

[0558] TaskAllocator: A module that distributes work orders

[0559] SupplementaryInfoProvider: A module that provides supplementary information

[0560] Data processing and calculation

[0561] The server uses a data collection module to collect information on participants' and workers' areas of expertise, proactiveness, past comments, and behavioral patterns. This data is analyzed by a comment allocation module, which provides appropriate opportunities to speak and work instructions at the appropriate time. After a comment is made or a task is completed, a reaction generation module generates positive feedback and provides it to the participant or worker. When there is silence or the work is stagnant, an information provision module provides relevant supplementary information. The emotion engine analyzes data obtained from the camera and microphone, recognizes the emotional state of the participants and workers, and responds accordingly.

[0562] Specific examples

[0563] For example, if you have employees working in a factory, the server might do the following:

[0564] 1. The data collection module collects information about employees' areas of expertise and past work history.

[0565] 2. Based on the data collected by TaskAllocator, work instructions are issued at the appropriate time.

[0566] 3. After the task is completed, the PositiveFeedbackGenerator provides positive feedback.

[0567] 4. When work stalls, SupplementaryInfoProvider provides relevant supplementary information.

[0568] 5. EmotionEngine recognizes employees' emotional state and responds accordingly.

[0569] Prompt Sentence Examples

[0570] "An employee is welding. Consider his past work history and areas of expertise to provide him with the next task instructions. Also, provide positive feedback after completing the task and provide relevant follow-up information when he stalls. Additionally, recognize his emotional state and respond accordingly."

[0571] In this way, the server, terminals, and users work together to improve the efficiency of meetings and work, and it becomes possible to maintain the motivation of participants and workers.

[0572] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0573] Step 1:

[0574] The server uses a data collection module to collect information on participants' and workers' areas of expertise, their proactiveness, past comments, and behavioral patterns. The inputs include video and audio data from cameras and microphones, as well as past work history data. This data is then analyzed to extract the characteristics of participants and workers. The output is feature data.

[0575] Step 2:

[0576] The server uses a speech allocation module to provide speaking opportunities at appropriate times based on the collected feature data. The inputs are feature data and data on the progress of the current meeting or work. The server analyzes this data and determines who should be given the opportunity to speak and when. The output is speech instruction data.

[0577] Step 3:

[0578] The server uses a reaction generation module to generate positive reactions after a comment or task completion. The server uses comment instruction data, comment content data, and task completion data as input. It analyzes these data and generates appropriate positive feedback. Feedback data is obtained as output.

[0579] Step 4:

[0580] The server uses an information provision module to provide supplementary information during periods of silence or stagnation. It uses current meeting and task progress data as input, analyzes this data, and generates relevant supplementary information. The output is the supplementary information data.

[0581] Step 5:

[0582] The server uses an emotion engine to recognize the emotional state of participants and workers and respond accordingly. It uses video and audio data from cameras and microphones as input. It analyzes this data to recognize the emotional state. The output is emotional state data.

[0583] Step 6:

[0584] The terminal displays the utterance instruction data, feedback data, supplementary information data, and emotional state data received from the server and provides them to the user. As input, it uses various data from the server. It displays this data in an appropriate format and provides it to the user. As output, it obtains information displayed in a format that is easy for the user to understand.

[0585] Step 7:

[0586] The user takes appropriate action based on the utterance instructions, feedback, supplementary information, and emotional state displayed on the device. The information displayed on the device is used as input. Based on this information, the next utterance or task is performed. The user's behavior is obtained as output.

[0587] Example 2

[0588] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0589] Conventional conference systems provide speaking opportunities without considering participants' areas of expertise or proactiveness, which often results in biased speech among certain participants. In addition, there is insufficient feedback on participants' comments, making it difficult to stimulate the meeting. Furthermore, there is no way to provide appropriate supplementary information during silences, which can stall the progress of the meeting. To solve these issues, a system is needed that takes into account the characteristics of participants, provides appropriate speaking opportunities, gives positive feedback, and provides necessary supplementary information.

[0590] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0591] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, and for initiating a conversation based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for recognizing participants' emotions, means for collecting participants' past comments and behavioral patterns, means for analyzing the collected data and extracting characteristics of participants, means for providing opportunities to speak based on the extracted characteristics, means for generating positive comments using a generative AI model, means for providing supplementary information using an information search engine, and means for understanding participants' emotional states using emotion recognition technology. This makes it possible to provide speaking opportunities, generate positive feedback, and provide appropriate supplementary information taking into account participants' characteristics.

[0592] "Participant" refers to an individual who takes part in a meeting or discussion.

[0593] "Areas of expertise" refers to areas in which participants have particular knowledge and experience and demonstrate outstanding abilities.

[0594] "Proactivity" refers to the attitude and tendency of participants to speak and act proactively in meetings and discussions.

[0595] "Number of comments" refers to the number of times a participant spoke during a meeting or discussion.

[0596] "Positive reactions" refer to providing positive feedback or responses to participants' comments or actions.

[0597] "Supplementary Information" refers to additional information or data relevant to the topic of the meeting or discussion.

[0598] "Emotion recognition" refers to technology that analyzes and identifies participants' emotional states from their facial expressions, voice, etc.

[0599] "Past comments" refers to comments made by participants in previous meetings or discussions.

[0600] "Behavioral patterns" refer to the behavioral tendencies and habits that participants have shown in the past.

[0601] "Data collection" refers to the process of collecting participants' past statements and behavioral patterns.

[0602] "Data analysis" refers to the process of analyzing collected data and extracting meaningful information.

[0603] "Characteristic extraction" refers to the process of identifying participants' characteristics, such as their areas of expertise and proactiveness, based on data analysis.

[0604] "Providing an opportunity to speak" means giving participants an opportunity to speak.

[0605] A "generative AI model" refers to a model that uses artificial intelligence to generate text and comments.

[0606] An "information search engine" refers to a system that searches for information on the Internet and obtains the necessary data.

[0607] "Emotion recognition technology" refers to technology for analyzing and identifying participants' emotional states.

[0608] MODE FOR CARRYING OUT THE INVENTION

[0609] This invention is a system that takes into account the characteristics of participants in meetings and discussions, provides appropriate opportunities to speak, gives positive feedback, and provides necessary supplementary information. Specific embodiments of this system are described below.

[0610] Data collection

[0611] The server collects participants' past comments and behavioral patterns. Specifically, it collects minutes of past meetings and participants' social media posts. This process uses an API to obtain the data. For example, it uses the Twitter API to collect participants' tweets.

[0612] Data analysis and feature extraction

[0613] The server analyzes the collected data and extracts characteristics such as participants' areas of expertise and proactiveness. This process uses natural language processing (NLP) techniques, specifically, analyzing text using the Python libraries NLTK and spaCy.

[0614] Providing opportunities to speak

[0615] The server provides speaking opportunities during the meeting based on the extracted characteristics. For example, when a topic on which participant A excels comes up, the server provides A with a speaking opportunity. Similarly, participant B, who speaks less, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up.

[0616] Generating positive reactions

[0617] The server generates positive reactions to participants' comments. For example, it generates positive comments such as "That's a fresh perspective" and provides feedback to participants. This process uses a generative AI model (e.g., OpenAI's GPT-3).

[0618] Providing supplementary information

[0619] The server provides supplementary information related to the topic. For example, if the topic is "new product development," it provides information on market trends, competitors' activities, etc. This process uses an information search engine or database.

[0620] Emotion recognition and response

[0621] The server uses an emotion engine that recognizes participants' emotions to understand their emotional state. For example, if a participant is nervous, it generates a comment to help them relax. This process uses emotion recognition technology (for example, Microsoft® Azure® Emotion API).

[0622] Specific examples

[0623] When a user starts a meeting, the server analyzes past meeting minutes and social media posts to extract the characteristics of the participants. For example, if participant A is found to have extensive knowledge of "marketing," the server will provide A with an opportunity to speak when a topic related to "marketing" comes up during the meeting. If participant B does not speak often, the server will provide him with an opportunity to speak when a topic related to B's area of ​​expertise comes up. Furthermore, the server generates positive comments such as "That's a fresh perspective" and provides feedback to the participants. If the topic is "new product development," the server will provide information such as market trends and the actions of competitors. The emotion engine then grasps the participants' emotional state and responds accordingly.

[0624] Prompt Sentence Examples

[0625] "Please explain how to give Participant A an opportunity to speak when a topic that he or she is good at comes up."

[0626] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0627] Step 1: Data collection

[0628] The server collects participants' past comments and behavioral patterns. Specifically, it uses an API to obtain minutes of past meetings and participants' social media posts. For example, it uses the Twitter API to collect participants' tweets. The input is the participants' identification information, and the output is the collected text data.

[0629] Step 2: Data analysis and feature extraction

[0630] The server analyzes the collected data and extracts characteristics such as participants' areas of expertise and proactiveness. This process uses natural language processing (NLP) technology. Specifically, it analyzes text using the Python libraries NLTK and spaCy. The input is the collected text data, and the output is participant characteristic information. The server tokenizes the text and extracts important keywords to identify participants' areas of expertise and proactiveness.

[0631] Step 3: Providing an opportunity to speak

[0632] The server provides speaking opportunities during the meeting based on the extracted characteristics. For example, when a topic that participant A is good at comes up on the agenda, the server provides A with a speaking opportunity. Similarly, participant B, who speaks less, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up on the agenda. The input is the participant's characteristic information and the progress of the meeting, and the output is a message offering a speaking opportunity. The server monitors the progress of the meeting in real time, and when a specific topic comes up on the agenda, it sends a message encouraging participants to speak.

[0633] Step 4: Generate positive reactions

[0634] The server generates positive reactions to participants' comments. For example, it generates positive comments such as "That's a fresh perspective" and provides them as feedback. This process uses a generative AI model (e.g., OpenAI's GPT-3). The input is the participant's comments, and the output is the generated positive comments. The server analyzes the participants' comments and sends prompts to GPT-3 to generate positive comments, which are then sent to the participants.

[0635] Step 5: Provide supporting information

[0636] The server provides supplementary information related to the agenda. For example, if the agenda is "new product development," it provides information such as market trends and competitors' activities. This process uses an information search engine and database. The input is the agenda content, and the output is related supplementary information. The server uses the Google Search API to obtain the latest information related to the agenda and provides it to participants during the meeting.

[0637] Step 6: Recognize and respond to emotions

[0638] The server uses an emotion engine that recognizes participants' emotions to grasp their emotional state. For example, if a participant is nervous, it generates a comment to help them relax. This process uses emotion recognition technology (for example, Microsoft Azure's Emotion API). The input is the participant's facial expressions and voice data, and the output is their emotional state and a corresponding response message. The server uses the Emotion API to identify the participant's emotional state, generates an appropriate response message, and sends it to the participant.

[0639] (Application example 2)

[0640] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0641] Conventional conference systems and work support systems have difficulty in taking into full consideration the characteristics and emotional states of participants and workers, making it difficult to achieve efficient communication and work support. Furthermore, there are problems with biased speaking opportunities and work delays, which can lead to lower overall efficiency. There is a need for a system that can solve these problems and respond appropriately to the characteristics of participants and workers.

[0642] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0643] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, and for initiating conversations based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating conversations, means for providing supplementary information related to the topic during silences, means including an emotion engine for recognizing participants' emotions, means for analyzing worker characteristics and issuing work instructions at appropriate times, means for generating positive comments and providing feedback when workers complete work, means for providing relevant supplementary information when work is delayed, and means for recognizing the emotional state of workers and responding accordingly. This enables appropriate responses according to the characteristics and emotional state of participants and workers, thereby achieving efficient communication and work support.

[0644] "Participant" refers to an individual who participates in a meeting or task.

[0645] "Area of ​​expertise" refers to an area in which a participant or worker has particular knowledge or skills.

[0646] "Proactivity" refers to the willingness and attitude of participants and workers to act voluntarily.

[0647] "Number of comments" refers to the number of times a participant spoke during a meeting or discussion.

[0648] "Participant characteristics" refers to individual characteristics of participants, such as their areas of expertise and proactiveness.

[0649] "Positive reactions" refers to providing positive feedback on the behavior of participants or workers.

[0650] "Silence" refers to a temporary halt in speech or activity during a meeting or task.

[0651] "Supplementary Information" refers to additional information related to an agenda or task.

[0652] An "emotion engine" refers to a system that recognizes the emotional state of participants and workers and responds accordingly.

[0653] "Worker" refers to an individual who performs work in a factory or work site.

[0654] "Work instructions" refers to giving specific work content and procedures to workers.

[0655] "Positive comments" refer to words that express positive evaluation of a worker's behavior or results.

[0656] "When work is delayed" refers to a situation where work is not progressing smoothly and stagnation or delays occur.

[0657] "Emotional state" refers to the current feelings or mood of a participant or worker.

[0658] The system for implementing this invention analyzes the characteristics of participants and workers, issues work instructions at appropriate times, provides positive feedback, and responds according to their emotional state. A specific embodiment of this system is described below.

[0659] Hardware and software used

[0660] Hardware:

[0661] Camera: Used to capture images of workers.

[0662] Robots: Used to provide work instructions and feedback.

[0663] software:

[0664] OpenAI API: Used to generate positive feedback and additional information.

[0665] EmotionRecognizer: Used to recognize the emotional state of workers.

[0666] Data processing and calculation

[0667] The server first analyzes the past comments and behavioral patterns of participants and workers to extract characteristics such as their areas of expertise and proactiveness. This analysis uses natural language processing technology and machine learning algorithms. Specifically, it analyzes meeting minutes and social media posts to extract participants' characteristics.

[0668] The server then issues work instructions at the appropriate time based on the extracted characteristics. For example, when a task that a worker excels at comes up, the server assigns that task to the worker. Also, when the work is stalled, the server provides relevant supplemental information. To achieve this, the server uses a generative AI model to input prompts such as:

[0669] Positive feedback generating prompt:

[0670] Generate positive feedback when workers do a great job.

[0671] Prompt statement for generating additional information:

[0672] Generate supplemental information to provide when work is delayed. Work: Machine maintenance

[0673] Furthermore, the server uses images captured by the camera to recognize the emotional state of the worker. Using EmotionRecognizer, it analyzes the worker's emotions in real time and takes appropriate action, such as encouraging them to take a break, if stress levels are high.

[0674] Specific examples

[0675] For example, imagine a worker performing machine maintenance in a factory. If the worker's past work history is analyzed and it is determined that the worker is skilled at machine operation, the server will instruct the worker to perform machine maintenance work. If the work is progressing smoothly, the server will provide positive feedback such as "Great job!". On the other hand, if the work is delayed, the server will provide supplemental information such as "This part of the maintenance requires special attention."

[0676] In addition, the system analyzes the facial expressions of workers through cameras and responds by telling them to take a short break if they are feeling stressed, thereby improving worker efficiency and satisfaction.

[0677] In this way, it becomes possible to respond appropriately to the characteristics and emotional state of participants and workers, enabling efficient communication and work support.

[0678] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0679] Step 1:

[0680] The server collects the past statements and behavioral patterns of participants and workers. Specifically, it collects data such as meeting minutes and social media posts. The input is data on past statements and behavioral patterns, and the output is the collected data.

[0681] Step 2:

[0682] The server analyzes the collected data and extracts characteristics of participants and workers, such as their areas of expertise and proactiveness. Specifically, it analyzes the data using natural language processing technology and machine learning algorithms. The input is the collected data, and the output is the extracted characteristic information.

[0683] Step 3:

[0684] The server generates work instructions at the appropriate time based on the extracted characteristic information. For example, when a task that a worker excels at arises, the server assigns that task to the worker. The input is the characteristic information and the current work status, and the output is the generated work instructions.

[0685] Step 4:

[0686] The server generates positive feedback when a worker completes a task. Specifically, the generative AI model inputs the following prompt sentence to generate a positive comment. The input is a notification of task completion, and the output is the generated positive comment.

[0687] Generate positive feedback when workers do a great job.

[0688] Step 5:

[0689] The server provides relevant supplemental information when work is delayed. Specifically, the generative AI model generates supplemental information by inputting the following prompt sentence. The input is a notification of work delay, and the output is the generated supplemental information.

[0690] Generate supplemental information to provide when work is delayed. Work: Machine maintenance

[0691] Step 6:

[0692] The server uses images acquired from the camera to recognize the emotional state of the worker. Specifically, it analyzes the images using EmotionRecognizer and recognizes the worker's emotions in real time. The input is the image acquired from the camera, and the output is the recognized emotional state.

[0693] Step 7:

[0694] The server responds according to the recognized emotional state, for example by generating a notification to take a break if stress levels are high. The input is the recognized emotional state, and the output is the generated response notification.

[0695] Example 3

[0696] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0697] In conventional conference systems, participants' opportunities to speak tend to be unevenly distributed, with certain participants often speaking up more than others. Furthermore, feedback on comments is insufficient, leading to participants feeling reluctant to speak up. Furthermore, silence during a meeting can slow down the progress of the meeting. There is a need to provide a conference system that solves these problems and allows all participants to speak up proactively.

[0698] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0699] In this invention, the server inputs characteristics such as participants' areas of expertise and proactiveness, and includes means for initiating a conversation based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for recognizing participants' emotions, and means for responding in accordance with participants' emotions. This allows all participants to have an equal opportunity to speak, reduces reluctance to speak, and makes it possible for the conference to proceed smoothly.

[0700] "Participant's areas of expertise" refer to areas in which individual participants in the conference have particular knowledge and experience and can specialize.

[0701] "Proactivity" refers to the willingness and attitude of participants to speak up and express their opinions voluntarily in a meeting.

[0702] The "number of times a participant speaks" refers to the number of times a participant speaks during a conference.

[0703] "Flooring techniques" refer to methods and techniques for providing specific participants with an opportunity to speak during a meeting.

[0704] A "positive reaction" is a reaction that motivates the speaker by providing positive comments or feedback in response to what the participant has said.

[0705] "Supplementary information" refers to additional information or data related to the meeting topic that provides participants with an opportunity to contribute new perspectives.

[0706] "Means of recognizing emotions" refers to techniques and methods for analyzing and understanding participants' emotional states from their facial expressions, voices, etc.

[0707] "Emotionally responsive methods" are methods and techniques for providing appropriate responses or feedback based on the perceived emotional state of participants.

[0708] The present invention is a system for encouraging participants to speak in a conference system and making the conference more lively. This system includes elements of a server, a terminal, and a user, and operates as follows.

[0709] Hardware and software used

[0710] Hardware

[0711] Server: A high-performance computer for running databases and generative AI models.

[0712] Devices: PCs, tablets, smartphones, etc. used by participants

[0713] Network equipment: Routers and switches for connecting servers and devices

[0714] software

[0715] Database management system: Software for storing and managing participant characteristics and meeting agendas

[0716] Generative AI model: An artificial intelligence model for analyzing speech content and generating positive reactions

[0717] Emotion Recognition Engine: Software for analyzing participants' emotional states

[0718] System Operation

[0719] Message sorting module

[0720] The server retrieves participant characteristic data (areas of expertise, number of times each participant speaks, proactiveness, etc.) from a database. Next, it analyzes the meeting agenda and determines which participant's areas of expertise each agenda item relates to. Based on the analysis results, the server distributes speaking opportunities equally among each participant. For example, when a topic that participant A is an expert on comes up, the server provides A with the opportunity to speak.

[0721] Reaction Generation Module

[0722] The server analyzes the content of participants' comments and generates positive reactions based on the content. Using a generative AI model, it generates positive comments in response to the comments and provides feedback to the speaker. For example, it generates a comment such as, "That's a fresh perspective."

[0723] Information Module

[0724] The server detects silence during a meeting. If no one speaks for a certain period of time, it determines that silence has occurred and retrieves supplementary information related to the agenda from a database or external information source. The retrieved supplementary information is provided to participants, giving them an opportunity to speak from a new perspective.

[0725] Emotion Engine

[0726] The server acquires participants' facial expressions and voice data and inputs it into an emotion engine. The emotion engine is used to analyze the participants' emotional state and respond accordingly based on the analysis results. For example, it provides comments to relax participants who are nervous.

[0727] Specific examples

[0728] Suppose the topic of a meeting is "new product development" and participant A is strong in marketing. The server provides A with an opportunity to speak when a topic related to marketing comes up. If participant B is strong in technology development but speaks infrequently, the server will provide B with an opportunity to speak when a topic related to technology development comes up. If silence occurs during a meeting, the server will provide supplementary information such as market trends and the actions of competitors. After a participant speaks, the server generates a positive reaction such as "That's a fresh perspective" and provides feedback to the speaker.

[0729] Prompt Sentence Examples

[0730] "In a meeting about new product development, if participant A is strong in marketing and participant B is strong in technical development but speaks less, explain how you would allocate speaking opportunities, what supplementary information to provide, what positive reactions to generate, and respond according to the emotions of the participants."

[0731] This system allows all participants to have an equal opportunity to speak, reduces reluctance to speak, and makes it possible to smoothly progress the conference. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0732] Step 1:

[0733] The server retrieves participant characteristic data (areas of expertise, number of comments, proactiveness, etc.) from a database. As input, the participant's ID and past comment history are required. The server analyzes this data and identifies each participant's areas of expertise and proactiveness. As output, the characteristic data of each participant is obtained.

[0734] Step 2:

[0735] The server analyzes the meeting agenda. As input, it requires the meeting agenda text. It uses natural language processing techniques to determine which participants' areas of expertise the agenda relates to. As output, it obtains mapping data between the agenda and participants' areas of expertise.

[0736] Step 3:

[0737] Based on the analysis results, the server allocates speaking opportunities equally to each participant. As input, it requires participant characteristic data and mapping data between the agenda and participants' areas of expertise. The server uses this data to execute an algorithm for allocating speaking opportunities. As output, it obtains data on the allocation of speaking opportunities to each participant.

[0738] Step 4:

[0739] The server analyzes the participants' comments. The input requires the participant's comment text. The server uses a generative AI model to analyze the comment and generate a positive reaction based on the content. The output is a positive comment.

[0740] Step 5:

[0741] The server feeds back the generated positive reactions to the speaker. As input, the positive comment and the speaker's ID are required. The server uses this data to send the comment to the speaker. As output, the feedback to the speaker is complete.

[0742] Step 6:

[0743] The server detects silence during a meeting. The input required is the audio data and speech logs of the meeting. The server determines silence when no speech is heard for a certain period of time. The output is the silence detection result.

[0744] Step 7:

[0745] The server retrieves supplementary information related to the agenda. As input, it requires the agenda text and access to external information sources. The server retrieves the supplementary information from a database or external information sources. As output, it obtains the supplementary information data.

[0746] Step 8:

[0747] The server provides the acquired supplementary information to the participants. As input, the supplementary information data and the participant's ID are required. The server uses these data to send the supplementary information to the participants. As output, the provision of information to the participants is complete.

[0748] Step 9:

[0749] The server acquires participants' facial expressions and voice data and inputs it into the emotion engine. The participants' facial expressions and voice data are required as input. The server uses the emotion engine to analyze the participants' emotional states. The emotion analysis results are obtained as output.

[0750] Step 10:

[0751] The server responds based on the participant's emotions based on the analysis results. The inputs required are the emotion analysis results and the participant's ID. The server uses this data to provide appropriate responses and feedback. The output is the completion of the emotional response to the participant.

[0752] (Application example 3)

[0753] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0754] In conventional conference systems, participants' opportunities to speak tend to be unevenly distributed, with certain participants often speaking more than others. Furthermore, feedback after speaking is insufficient, which can lead to participants feeling reluctant to speak up. Furthermore, when there are silences in the meeting, appropriate supplementary information is not provided, which can slow down the progress of the meeting. There is a need to solve these problems and make meetings more lively.

[0755] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0756] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, and initiating a conversation based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for recognizing participants' emotions and responding accordingly, means for analyzing the meeting agenda and allocating speaking opportunities based on participants' areas of expertise, means for generating positive comments after participants speak and providing feedback to the speaker, means for providing supplementary information related to the agenda during silences in the meeting, and means for capturing participants' facial expressions and analyzing them with an emotion recognition engine.This encourages participants to speak and makes the meeting more lively.

[0757] "Participant's areas of expertise" are areas in which each participant in the conference has specialized knowledge or experience.

[0758] "Positivity" refers to the willingness and attitude of participants to speak and act voluntarily.

[0759] "Number of remarks" refers to the number of times a participant spoke during a meeting.

[0760] "Participant characteristics" refers to information such as participants' areas of expertise, their proactiveness, past comments, and behavioral patterns.

[0761] "Methods of initiating conversation" are methods of providing participants with opportunities to speak at appropriate times based on their characteristics and the number of times they have spoken.

[0762] "Positive reactions" refer to making positive comments or feedback in response to participants' comments.

[0763] "Supplementary Information" refers to additional information or data related to the agenda.

[0764] "Emotion recognition" refers to analyzing a participant's emotional state from their facial expressions, tone of voice, etc.

[0765] "Analyzing the agenda" refers to understanding the theme or topic of the meeting and extracting information to respond appropriately.

[0766] "Allocating speaking opportunities" refers to adjusting the order and timing of speaking, taking into consideration the participants' areas of expertise and the number of times they will speak.

[0767] "Positive comments" refer to providing positive evaluations or encouraging words in response to participants' comments.

[0768] "Capturing facial expressions" refers to using a camera or other device to capture a participant's facial expressions.

[0769] An "emotion recognition engine" is software that analyzes captured facial expressions and voice data to determine the emotional state of participants.

[0770] The system for carrying out the present invention supports meetings using factory robots. The specific configuration and operation of the system will be described below.

[0771] System configuration

[0772] 1. Hardware:

[0773] Factory robots: Examples include humanoid robots such as Pepper and NAO.

[0774] Microphone and speaker: Used to capture what participants say and provide audio feedback.

[0775] Camera: Used to capture participants' facial expressions and perform emotion recognition.

[0776] 2. Software:

[0777] Speech recognition software: We will use Google Speech-to-Text as an example.

[0778] Emotion Recognition Engine: Using Affectiva as an example.

[0779] Natural language processing engine: Using GPT-4 as an example.

[0780] Database: Used to store participant characteristics and speech history.

[0781] System Operation

[0782] 1. Speech Recognition:

[0783] A microphone captures what participants say, which is then converted into text using speech recognition software, which then stores the speech as digital data.

[0784] 2. Participant characteristics input:

[0785] Participants' areas of expertise and number of comments are stored in a database and loaded at the start of the meeting, allowing the characteristics of each participant to be input into the system.

[0786] 3. Message allocation:

[0787] It uses a natural language processing engine to analyze the agenda and allocate speaking opportunities based on participants' areas of expertise. For example, when a technical topic comes up, it provides speaking opportunities to technical experts.

[0788] 4. Generating positive reactions:

[0789] After a participant speaks, a natural language processing engine is used to generate positive comments and send them through the speaker, helping participants feel less reluctant to speak up.

[0790] 5. Supplementary information provided:

[0791] When a meeting falls silent, the system retrieves supplementary information related to the agenda from a database, formats it appropriately using a natural language processing engine, and provides it, for example, information such as "Current market trends are as follows."

[0792] 6. Emotion recognition:

[0793] The camera captures participants' facial expressions and analyzes them with an emotion recognition engine. Based on the analysis results, appropriate responses are taken. For example, if a participant is nervous, comments to help them relax are generated.

[0794] Examples and prompts

[0795] Specific examples

[0796] Meeting scenario:

[0797] Agenda: New product development

[0798] Participants: Mechanical Engineer A, Marketing Officer B, Quality Control Officer C

[0799] Message sorting:

[0800] When a technical topic that mechanical engineer A is good at comes up, A is given an opportunity to speak.

[0801] Positive reactions:

[0802] After Mechanical Engineer A makes a statement, a comment such as "That's a fresh perspective" is generated.

[0803] Additional information provided:

[0804] When the meeting falls silent, provide information such as, "Here are the current market trends."

[0805] Prompt statement

[0806] "In meetings about new product development, give Mechanical Engineer A the opportunity to speak when technical topics come up and generate positive reactions after he speaks. Also, provide additional information about market trends when the meeting falls silent."

[0807] In this way, factory robots can support meetings, encouraging participants to speak up and making the meeting more lively.

[0808] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0809] Step 1:

[0810] Participant characteristics input

[0811] At the start of a meeting, the server reads the characteristics of participants, such as their areas of expertise and number of comments, from a database. The input includes the participants' past comments and behavioral patterns. Based on this data, the characteristics of the participants are extracted and input into the system. The output is the characteristics of each participant, such as their areas of expertise and proactiveness.

[0812] Step 2:

[0813] Voice Recognition

[0814] The terminal (factory robot) uses a microphone to capture what the participant says. The input contains the participant's voice data. It uses speech recognition software (e.g., Google Speech-to-Text) to convert the voice data into text. The output is the text data of what was said.

[0815] Step 3:

[0816] Speech allocation

[0817] The server analyzes the agenda using a natural language processing engine (e.g., GPT-4). The input includes the text data of the agenda and the characteristics of the participants. Based on the analysis results, it allocates speaking opportunities to participants according to their areas of expertise. The output is a list of participants to whom speaking opportunities will be provided.

[0818] Step 4:

[0819] Generate positive reactions

[0820] The server uses a natural language processing engine to generate positive comments after participants make comments. The input includes text data of the comments. The generated comments are fed back to the speaker via the speaker. The output is text data of the positive comments.

[0821] Step 5:

[0822] Supplementary information provided

[0823] The server retrieves supplementary information related to the agenda from the database when there is silence in the meeting. The input includes the text data of the agenda. The server uses a natural language processing engine to format the supplementary information appropriately and provides it. The output is the text data of the supplementary information.

[0824] Step 6:

[0825] emotion recognition

[0826] The terminal (factory robot) uses a camera to capture the participant's facial expression. The input contains the participant's facial expression data. An emotion recognition engine (e.g., Affectiva) is used to analyze the facial expression data and determine the participant's emotional state. The output is the emotional state data.

[0827] Step 7:

[0828] Responding according to emotions

[0829] The server then responds appropriately based on the analysis results of the emotion recognition engine. The input includes data on the participant's emotional state. For example, if a participant is nervous, the server generates a comment to help them relax. The output is text data for the appropriate response.

[0830] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0831] The data generation model 58 is a so-called generative AI (Artificial Intelligence).

[0832] An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0833] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0834] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0835] [Second embodiment]

[0836] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0837] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0838] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0839] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0840] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0841] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0842] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0843] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0844] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0845] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0846] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0847] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0848] "Example 1"

[0849] The present invention is comprised of a data collection module for grasping the characteristics of conference participants, a speech allocation module for providing speaking opportunities, a reaction generation module for generating positive reactions after speaking, and an information provision module for providing supplementary information during silences.

[0850] "Example 2"

[0851] The data collection module analyzes participants' past comments and behavioral patterns to extract characteristics such as their areas of expertise and proactiveness. For example, it is possible to extract participant characteristics by analyzing past meeting minutes and participants' social media posts.

[0852] "Example 3"

[0853] The speech allocation module distributes speaking opportunities equally based on the extracted participant characteristics and the meeting agenda. For example, when a topic on which participant A excels comes up, the opportunity to speak is provided to A. Similarly, participant B, who speaks less frequently, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up.

[0854] "Example 4"

[0855] The reaction generation module generates positive reactions and provides feedback to participants after they speak. For example, it generates positive comments such as, "That's a fresh perspective." This helps participants feel less reluctant to speak up.

[0856] "Example 5"

[0857] The information module provides supplementary information related to the agenda when there is silence in the meeting. For example, if the agenda is "new product development," it can provide information on market trends and competitors' activities. This gives participants an opportunity to speak up from a new perspective.

[0858] The processing flow of each embodiment will be described below.

[0859] "Example 1"

[0860] Step 1: The data collection module runs and analyzes participants' past comments and behavioral patterns. This analysis extracts characteristics such as participants' areas of expertise and proactiveness.

[0861] Step 2: The speech allocation module operates and allocates speaking opportunities equally based on the extracted participant characteristics and the meeting agenda.

[0862] Step 3: The reaction generation module operates to generate positive reactions after a participant speaks and provide feedback to the speaker.

[0863] Step 4: The information provision module operates to provide supplementary information related to the agenda when the meeting is silent.

[0864] "Example 2"

[0865] Step 1: The data collection module runs and analyzes participants' past meeting minutes, their social media posts, etc. This analysis extracts the participants' characteristics.

[0866] Step 2: The speech allocation module operates, and when a topic that Participant A is good at comes up, it provides Participant A with an opportunity to speak. Also, Participant B, who has not spoken often, is provided with an opportunity to speak when a topic related to Participant B's area of ​​expertise comes up.

[0867] Step 3: The reaction generation module operates and generates positive comments such as "That's a fresh perspective" and provides feedback to the participants.

[0868] Step 4: The information provision module operates and, if the topic is "new product development," provides information such as market trends and competitors' activities.

[0869] Example 1

[0870] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0871] Conventional conference systems have difficulty fully understanding the characteristics of participants and providing appropriate opportunities to speak. Furthermore, they lack positive reactions after speaking and supplementary information during silences, making it difficult to facilitate smooth progress in meetings or encourage active participation. This has led to problems such as reduced meeting efficiency and productivity.

[0872] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0873] In this invention, the server includes a means for collecting participant profile information and storing it in a database, a means for generating positive reactions using a generative AI model, and a means for searching for and providing related supplemental information using the generative AI model. This makes it possible to understand the characteristics of participants and provide appropriate opportunities to speak. In addition, by providing positive reactions after speaking and supplemental information during silences, the meeting can proceed smoothly and encourage active participation by participants.

[0874] "Participants' specialties, proactiveness, and other characteristics" refers to the characteristics of each person participating in the meeting, such as their expertise and skills, frequency of speaking, and motivation.

[0875] "Number of speeches" refers to the number of times each participant spoke during the conference.

[0876] A "speaking device" refers to a method or system for providing a specific participant with an opportunity to speak during a meeting.

[0877] "Means for positive reactions" refers to methods or systems for showing positive reactions or evaluations after participants make comments.

[0878] "Means for providing supplementary information related to the topic during silence" refers to a method or system for providing additional information related to the topic when silence occurs during a meeting.

[0879] "Profile Information" refers to personal information about you, such as your name, job title, and area of ​​expertise.

[0880] A "database" refers to an information management system that systematically stores collected information and enables it to be searched and used as needed.

[0881] A "generative AI model" refers to a model that has been trained using artificial intelligence techniques to perform a specific task.

[0882] "Supplementary information" refers to additional data or knowledge relevant to the meeting topic.

[0883] "Means for searching and providing" refers to a method or system for searching the Internet or a database for necessary information and providing it to the user.

[0884] MODE FOR CARRYING OUT THE INVENTION

[0885] This invention is a system that understands the characteristics of conference participants, provides appropriate opportunities to speak, generates positive reactions after speaking, and provides supplementary information during silences. This system works in cooperation with three parties: a server, a terminal, and a user.

[0886] Data Collection Module

[0887] 1. A user logs in to a terminal to participate in a conference.

[0888] 2. The device displays a screen for entering the user's profile information.

[0889] 3. The user enters profile information such as name, job title, and area of ​​expertise.

[0890] 4. The terminal sends the entered information to the server.

[0891] 5. The server stores the received profile information in a database.

[0892] Examples:

[0893] The user enters their name and job title into a form displayed on the terminal screen.

[0894] The terminal transmits the input information to the server in real time.

[0895] The server stores the data in the database in the format "User A: Name = Yamada Taro, Position = Manager, Area of ​​Expertise = Marketing."

[0896] Message sorting module

[0897] 1. The server monitors the progress of the conference and schedules speaking opportunities for each participant.

[0898] 2. The server selects appropriate participants for a particular topic based on the collected data.

[0899] 3. The terminal receives instructions from the server and notifies the selected participants when it is time to speak.

[0900] 4. The user receives the notification and makes a comment.

[0901] Examples:

[0902] The server executes logic such as "encourage Yamada Taro, who has expertise in marketing, to speak on agenda item A."

[0903] The device displays the message "Please speak next, Yamada Taro."

[0904] The user confirms the notification and starts speaking.

[0905] Reaction Generation Module

[0906] 1. The server analyzes the user's comments in real time.

[0907] 2. The server inputs a prompt sentence into the generative AI model to generate a positive reaction.

[0908] 3. The device displays the generated reaction to the user.

[0909] 4. The user checks the displayed reaction.

[0910] Examples:

[0911] The server inputs a prompt statement into the generative AI model: "Analyze what User A said and generate a positive reaction."

[0912] The generative AI model generates the reaction, "That's a great idea, Yamada!"

[0913] The device displays this reaction on the screen.

[0914] The user sees the message, "Great idea, Yamada!"

[0915] Information Module

[0916] 1. The server searches for relevant supplemental information during silences in the meeting.

[0917] 2. The server collects relevant data and selects the appropriate information.

[0918] 3. The terminal displays the supplementary information sent from the server to the user.

[0919] 4. The user checks the displayed supplementary information.

[0920] Examples:

[0921] The server inputs the prompt statement "Find the latest news articles related to the current topic" into the generative AI model.

[0922] Generative AI models search for relevant news articles and statistical data and generate information such as, "Here's an article about the latest marketing trends."

[0923] The terminal displays this information on the screen.

[0924] The user sees a message that reads, "Here's an article about the latest marketing trends."

[0925] Prompt Sentence Examples

[0926] "Analyze what User A said and generate a positive reaction."

[0927] "Find the latest news articles related to your current topic."

[0928] The above is an embodiment of the present invention.

[0929] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0930] Step 1:

[0931] A user logs in to a terminal to participate in a conference.

[0932] Input: User login information (user ID, password)

[0933] Output: Login success or failure message

[0934] Specific operation: The user enters their user ID and password into the login screen on the device and clicks the login button. The device sends the entered information to the server, which then performs authentication. If authentication is successful, a login success message is displayed.

[0935] Step 2:

[0936] The terminal displays a screen for inputting user profile information.

[0937] Input: None

[0938] Output: Profile information input screen

[0939] Specific operation: After successfully logging in, the terminal will display a form for the user to enter profile information. The form includes input fields such as name, job title, and area of ​​expertise.

[0940] Step 3:

[0941] Users enter profile information such as name, job title, and area of ​​expertise.

[0942] Input: Profile information such as name, job title, area of ​​expertise, etc.

[0943] Output: Entered profile information

[0944] Specific operation: The user enters information such as name, job title, and area of ​​expertise into the form on the terminal and clicks the submit button.

[0945] Step 4:

[0946] The terminal transmits the input information to the server.

[0947] Input: Profile information entered by the user

[0948] Output: Result sent to the server

[0949] Specific operation: The device sends the profile information entered by the user to the server, which then verifies the received information and stores it in a database.

[0950] Step 5:

[0951] The server stores the received profile information in a database.

[0952] Input: Profile information sent from the device

[0953] Output: Save results to database

[0954] Specific operation: The server saves the received profile information in the database. If the save is successful, it generates a save success message.

[0955] Step 6:

[0956] The server monitors the progress of the conference and schedules speaking opportunities for each participant.

[0957] Input: Meeting progress, participant profile information

[0958] Output: Speaking Opportunity Plan

[0959] Specific operation: The server plans appropriate speaking opportunities for each participant based on the progress of the meeting and the participant's profile information. The plan includes who will speak on which topic.

[0960] Step 7:

[0961] The server uses the collected data to select appropriate participants for a particular topic.

[0962] Input: Participant profile information, meeting agenda

[0963] Output: Selected participants

[0964] Specific operation: The server analyzes participants' areas of expertise and past comments and selects the most appropriate participants for a particular topic.

[0965] Step 8:

[0966] The terminal receives instructions from the server and notifies the selected participants of the timing to speak.

[0967] Input: Instructions from the server (selected participants, timing to speak)

[0968] Output: Notification of speaking timing

[0969] Specific operation: The terminal receives instructions from the server and displays the message "Please speak next" to the selected participant.

[0970] Step 9:

[0971] The user receives the notification and makes a comment.

[0972] Input: Notification of timing to speak

[0973] Output: What the user said

[0974] Specific operation: The user checks the notification displayed on the device and speaks at the specified timing.

[0975] Step 10:

[0976] The server analyzes the content of the user's comments in real time.

[0977] Input: What the user says

[0978] Output: Analysis results

[0979] Specific operation: The server analyzes the text of the user's comments and extracts important keywords and context.

[0980] Step 11:

[0981] The server inputs prompt sentences into the generative AI model to generate positive reactions.

[0982] Input: Analysis results, prompt text

[0983] Output: Positive reaction

[0984] Specific operation: The server inputs the prompt "Analyze what User A said and generate a positive reaction" into the generative AI model. The generative AI model generates the reaction "That's a great idea, Yamada-san!"

[0985] Step 12:

[0986] The terminal displays the generated reaction to the user.

[0987] Input: Positive reaction

[0988] Output: Show reactions

[0989] Specific behavior: The device displays the generated positive reaction on the screen. The user sees the message, "That's a great idea, Yamada-san!"

[0990] Step 13:

[0991] The server searches for relevant supplemental information during periods of silence in the conference.

[0992] Input: Meeting agenda, silence occurrence

[0993] Output: Supplementary information

[0994] How it works: The server inputs a prompt to the generative AI model: "Find the latest news articles related to the current topic." The generative AI model searches for relevant news articles and statistical data and generates information such as, "Here's an article about the latest marketing trends."

[0995] Step 14:

[0996] The terminal displays the supplementary information sent from the server to the user.

[0997] Input: Supplementary information

[0998] Output: Display supplementary information

[0999] Specific operation: The device displays the supplementary information sent from the server on the screen. The user sees the message "Click here for an article about the latest marketing trends."

[1000] (Application example 1)

[1001] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1002] Conventional conference and communication systems have difficulty fully understanding the characteristics of participants and providing opportunities to speak at appropriate times. Furthermore, they often lack sufficient support for reactions after a comment or supplementary information during silences, hindering smooth communication. Furthermore, in the case of communication between workers and robots in factories, there is a lack of instructions and feedback that take into account the characteristics of the workers, resulting in a decline in work efficiency.

[1003] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1004] In this invention, the server includes means for inputting characteristics of participants such as their areas of expertise and proactiveness, and for initiating conversations based on the number of times participants speak and their characteristics, means for generating positive reactions after initiating conversations, means for providing supplementary information related to the topic during silences, means for collecting characteristic data of workers, means for issuing work instructions at appropriate times, means for generating positive reactions to comments made by workers, and means for providing supplementary information when work is stalled. This enables smooth communication and efficient work instructions that take into account the characteristics of participants and workers.

[1005] "Participant" refers to an individual or group participating in a meeting or task.

[1006] "Area of ​​expertise" refers to a participant's area of ​​particular proficiency or expertise.

[1007] "Proactivity" refers to the attitude and character of participants to act spontaneously and actively express their opinions and ideas.

[1008] "Number of remarks" refers to the number of times a participant spoke during a meeting or task.

[1009] "Methods of starting a conversation" refers to methods and techniques for providing participants with an opportunity to speak and starting a topic at an appropriate time.

[1010] "Positive reactions" refer to having a positive reaction to a participant's words or actions.

[1011] "Silent times" refer to periods when no one is speaking during a meeting or during work.

[1012] "Supplementary information" refers to additional information or explanation related to a topic or task.

[1013] "Worker" refers to an individual who actually performs work in a factory or work site.

[1014] "Characteristic data" refers to data that indicates a worker's characteristics, such as their areas of expertise, experience, and behavioral patterns.

[1015] "Work instructions" refers to giving specific work content and procedures to a worker.

[1016] "Means for generating" refers to methods or technologies for automatically producing specific reactions or information.

[1017] A system for implementing this invention includes means for inputting characteristics such as areas of expertise and proactiveness of participants, and for initiating conversation based on the number of times participants speak and their characteristics, means for generating positive reactions after initiating conversation, means for providing supplementary information related to the topic during periods of silence, means for collecting data on worker characteristics, means for issuing work instructions at appropriate times, means for generating positive reactions to comments made by workers, and means for providing supplementary information when work is stalled.

[1018] The server runs a Python program to collect and analyze data on the characteristics of participants and workers. Specifically, it analyzes participants' past comments and behavioral patterns to extract their areas of expertise and proactiveness. This allows it to allocate conversations in a way that evens out the number of times each participant speaks.

[1019] The server also uses a generative AI model (e.g., GPT-3) to generate positive reactions to participants' comments. For example, if a participant asks, "What's the next step?", the server inputs the following prompt into the generative AI model:

[1020] The participant asked, "What's the next step?" Provide a positive reaction and supporting information.

[1021] An example of the output from a generative AI model is the following response:

[1022] Great question! The next step is to attach part A to part B. The key to this step is to check the orientation of part A.

[1023] Additionally, the server provides topic-related supplemental information during silences. For example, if no one is speaking during a meeting, the server uses a generative AI model to provide supplemental information such as:

[1024] The next step in this process is to make sure that part A is oriented correctly when attaching it to part B.

[1025] In this way, the server can realize smooth communication and efficient work instructions that take into account the characteristics of participants and workers. Potential hardware to be used includes robots in factories and smart glasses worn by workers. The software uses Python programs and generative AI models (e.g., GPT-3).

[1026] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1027] Step 1:

[1028] The server collects characteristic data of participants and workers. Specifically, it analyzes participants' past comments and behavioral patterns to extract their areas of expertise and proactiveness. It uses participants' past comment records and behavior logs as input and generates data on participants' areas of expertise and proactiveness as output.

[1029] Step 2:

[1030] The server uses the collected characteristic data to initiate a conversation based on the number of times participants have spoken and their characteristics. Specifically, it initiates a topic to even out the number of times each participant has spoken. As input, it uses the participant's characteristic data and the current number of times they have spoken, and as output, it determines the participant who should speak next.

[1031] Step 3:

[1032] The server generates a positive reaction after initiating the conversation. Specifically, it uses a generative AI model (e.g., GPT-3) to generate a positive reaction to the participant's comments. It uses the participant's comments as input and generates a positive reaction as output.

[1033] Step 4:

[1034] The server provides topic-related supplemental information during silences. Specifically, it uses a generative AI model to generate supplemental information related to meetings and tasks. It uses the current topic and task content as input and generates supplemental information as output.

[1035] Step 5:

[1036] The server collects the characteristic data of the workers and issues work instructions at the appropriate time. Specifically, it generates optimal work instructions based on the worker's expertise and experience. The server uses the worker's characteristic data and current work situation as input and generates work instructions as output.

[1037] Step 6:

[1038] The server generates a positive reaction to the worker's utterance. Specifically, it uses a generative AI model to generate a positive reaction to the worker's utterance. It uses the worker's utterance as input and generates a positive reaction as output.

[1039] Step 7:

[1040] The server provides supplementary information when a task stalls. Specifically, it uses a generative AI model to generate supplementary information related to the task. It uses the current task status as input and generates the supplementary information as output.

[1041] Example 2

[1042] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1043] Conventional conference systems have the problem of reducing meeting efficiency because they initiate conversations without considering participants' areas of expertise or proactiveness. Furthermore, there is a lack of means to analyze participants' past comments and behavioral patterns to extract their characteristics, making it difficult to provide appropriate topics or positive reactions. Furthermore, there are also insufficient means to provide relevant supplementary information during silences.

[1044] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1045] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, means for initiating a conversation based on the number of times participants have spoken and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for collecting participants' past speech content and behavioral patterns, means for preprocessing the collected data, means for extracting characteristics from the preprocessed data, means for evaluating the extracted characteristics, and means for outputting the evaluation results. This enables efficient conference management that takes into account the characteristics of participants.

[1046] "Participants' specialties, proactiveness, and other characteristics" are characteristics that indicate expertise and skills in specific fields, as well as proactiveness in meetings and discussions, extracted from participants' past statements and behavioral patterns.

[1047] "Methods of starting a conversation based on the number of times participants have spoken and their characteristics" refers to a method of providing topics at appropriate times and encouraging participants to speak based on the number of times participants have spoken and their characteristics.

[1048] "Means for a positive reaction" are ways to positively evaluate a participant's comments after they have been made and to increase their motivation.

[1049] "Means for providing additional information related to the topic during silence" refers to methods for providing additional information related to the topic and stimulating conversation when silence occurs during a meeting.

[1050] "Means of collecting participants' past statements and behavioral patterns" refers to methods for collecting data on participants' past statements and behavior, such as meeting minutes and social media posts.

[1051] The "means for preprocessing collected data" refers to a method for performing preprocessing such as removing unnecessary information and tokenizing text to make the collected data easier to analyze.

[1052] The "means for extracting characteristics from preprocessed data" is a method for extracting characteristics such as participants' areas of expertise and proactiveness based on the preprocessed data.

[1053] The "means for evaluating extracted characteristics" is a method for evaluating participants' areas of expertise and proactiveness based on the extracted characteristics, and for using this information to help manage the meeting.

[1054] "Means for outputting evaluation results" refers to a method for outputting the results of the evaluated characteristics in JSON format or the like and sending them to other systems or terminals.

[1055] The present invention is a system for efficiently managing a conference based on the characteristics of participants, such as their areas of expertise and their proactiveness. A specific embodiment of this system will be described below.

[1056] 1. Generating the system program

[1057] The server generates a system program that includes a data collection module. This program analyzes participants' past comments and behavioral patterns, and extracts their characteristics, such as their areas of expertise and proactiveness.

[1058] 2. Explanation of program processing

[1059] The server processes and calculates data using the following hardware and software:

[1060] Hardware and software used

[1061] Hardware: Servers, terminals

[1062] Software: Python, NLTK, SpaCy, scikit-learn, TensorFlow, Twitter API, Google Drive API

[1063] Data collection

[1064] The server retrieves participants' social media posts using the Twitter API, and the meeting minutes using the Google Drive API.

[1065] Data Preprocessing

[1066] The server uses the NLTK library to clean and tokenize the text data, for example splitting the text "I'm good at project management" into "project," "management," and "good at."

[1067] Feature Extraction

[1068] The server uses TF-IDF (Term Frequency-Inverse Document Frequency) to extract important keywords such as "project management" and "leadership." For example, if the TF-IDF score of "project management" is high, the server determines that the word is important.

[1069] Characterization

[1070] The server uses a random forest model from scikit-learn to evaluate participants' areas of expertise and proactiveness. For example, if a participant has a high score in "project management," the server evaluates the participant as being good at project management.

[1071] Output of results

[1072] The server outputs the evaluation results in JSON format and sends them to the terminal. The terminal displays the evaluation results to the user. For example, it displays the result that "Participant A is good at project management and is proactive."

[1073] 3. Examples of concrete examples and prompts

[1074] For example, if keywords such as "project management" and "leadership" appear frequently in the minutes of a meeting, the server will determine that the participant is skilled in project management. Similarly, if phrases such as "active exchange of opinions" and "proposing new ideas" appear frequently in social media posts, the participant will be evaluated as proactive.

[1075] Example prompts to input to the generative AI model:

[1076] "Evaluate participants' areas of expertise and proactiveness based on past meeting minutes and social media posts."

[1077] In this way, the server performs a series of processes from data collection to output of results, realizing efficient conference management that takes into account the characteristics of the participants.

[1078] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1079] Step 1: Data collection

[1080] The server collects participants' past comments and behavioral patterns. Specifically, it uses the Twitter API to obtain participants' social media posts and the Google Drive API to obtain meeting minutes. The input is raw data obtained through the API, and the output is the collected text data.

[1081] Step 2: Data Preprocessing

[1082] The server preprocesses the collected data. Specifically, it uses the NLTK library to clean and tokenize the text data. For example, it splits the text "I'm good at project management" into "project," "management," and "good at." The input is the collected text data, and the output is the preprocessed, clean text data.

[1083] Step 3: Feature extraction

[1084] The server extracts features from the preprocessed data. Specifically, it uses TF-IDF (Term Frequency-Inverse Document Frequency) to extract important keywords and phrases. For example, if the TF-IDF score of "project management" is high, it determines that the word is important. The input is the preprocessed clean text data, and the output is the extracted keywords and phrases.

[1085] Step 4: Characterization

[1086] The server evaluates participants' characteristics, such as their areas of expertise and proactiveness, based on the extracted keywords and phrases. Specifically, it uses a random forest model from scikit-learn to learn from past data and predict characteristics for new data. The input is the extracted keywords and phrases, and the output is a score for the evaluated characteristics.

[1087] Step 5: Output the results

[1088] The server outputs the evaluation results in JSON format and sends them to the terminal. The terminal displays the evaluation results to the user. For example, it displays the result "Participant A is good at project management and is proactive." The input is the score of the evaluated characteristics, and the output is the evaluation result displayed to the user.

[1089] (Application example 2)

[1090] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1091] Conventional systems have the problem of reducing meeting and work efficiency because they initiate conversations without considering the areas of expertise or enthusiasm of the participants. Also, when allocating work within a factory, they assign tasks without considering the areas of expertise or work efficiency of the workers, which reduces production efficiency. To solve these problems, a system is needed that accurately grasps the characteristics of participants and workers, and based on that, can initiate conversations and allocate work optimally.

[1092] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1093] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, and for initiating conversations based on the number of comments and participant characteristics, means for providing positive reactions after initiating conversations, means for providing supplementary information related to the topic during silences, means for analyzing the past work history and behavioral patterns of workers to identify their areas of expertise and work efficiency, means for automatically assigning work based on their areas of expertise, and means for providing appropriate feedback when a decline in work efficiency is observed. This allows for an accurate understanding of the characteristics of participants and workers, and enables optimal conversation initiation and task assignment based on this information.

[1094] "Participant" refers to an individual who participates in a meeting or task.

[1095] "Area of ​​expertise" refers to an area in which a participant or worker has particularly outstanding abilities or knowledge.

[1096] "Proactivity" refers to the attitude of participants and workers to act voluntarily and take on a proactive approach.

[1097] "Number of times spoken" refers to how often participants speak in meetings or communication settings.

[1098] "Features" refer to individual characteristics extracted from the behavioral patterns and past statements of participants and workers.

[1099] "Starting a conversation" refers to the act of providing a topic to a specific participant and encouraging them to speak.

[1100] "Positive reactions" refer to the act of showing a positive response to the comments or actions of other participants.

[1101] "Silence" refers to a time when no one is speaking during a meeting or communication session.

[1102] "Supplemental information" refers to additional information or explanation related to the topic.

[1103] "Work history" refers to a record of work that a worker has done in the past.

[1104] "Behavioral patterns" refer to the past behavioral tendencies and habits of participants or workers.

[1105] "Work efficiency" refers to an index that indicates how efficiently a worker performs work within a certain period of time.

[1106] "Work assignment" refers to the act of assigning a specific task to a specific worker.

[1107] "Feedback" refers to evaluations and advice provided to workers regarding their behavior and performance.

[1108] The system for implementing this invention consists of three main elements: a server, a terminal, and a user. The server includes a data collection module that analyzes the past comments and behavioral patterns of participants and workers to extract characteristics such as areas of expertise and proactiveness. The terminal is a tablet or smartphone used by the worker, and provides an interface for receiving feedback and work assignments from the server. The user is an individual who participates in meetings and work, and acts based on the information provided by the system.

[1109] The server processes and calculates data using software such as Python, Pandas, and Scikit-learn. Specifically, it vectorizes text data using TfidfVectorizer and analyzes the data using KMeans clustering. This allows for an accurate understanding of the characteristics of participants and workers, and allows for optimal topic assignment and task allocation based on that information.

[1110] For example, the server may analyze the past work history of worker A and determine that he is good at welding. It may also analyze the social media posts of worker B and determine that he has a tendency to proactively try new tasks. Based on this, the server may assign welding work to worker A and new work to worker B. Furthermore, if a decline in work efficiency is observed, the server may provide appropriate feedback.

[1111] Examples of prompts using generative AI models include:

[1112] "Analyze worker A's past work history and identify the tasks he excels at. Also, analyze worker B's social media posts and evaluate his proactiveness."

[1113] In this way, a system can be realized that accurately grasps the characteristics of participants and workers and, based on that, optimally initiates topics and assigns tasks.

[1114] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1115] Step 1:

[1116] The server collects the worker's past work history data and SNS data.

[1117] Input: Work history data (CSV file), SNS data (CSV file)

[1118] Output: Work history data frame, SNS data frame

[1119] Specific operation: The server reads the work history data and SNS data from the CSV file and converts them into a data frame using Pandas.

[1120] Step 2:

[1121] The server analyzes the work history data and identifies the worker's specialty.

[1122] Input: Work history data frame

[1123] Output: Specialty clusters for each worker

[1124] Specific operation: The server vectorizes the work history data using TfidfVectorizer and identifies each worker's areas of expertise using KMeans clustering.

[1125] Step 3:

[1126] The server analyzes the SNS data and evaluates the proactiveness of the worker.

[1127] Input: SNS data frame

[1128] Output: Proactivity clusters for each worker

[1129] Specific operation: The server vectorizes the SNS data using TfidfVectorizer and evaluates the proactiveness of each worker using KMeans clustering.

[1130] Step 4:

[1131] The server automatically assigns work based on expertise and proactiveness.

[1132] Input: Specialty cluster and proactiveness cluster for each worker

[1133] Output: Work assignment list

[1134] Specific operation: The server executes an algorithm to assign the most suitable tasks to each worker based on cluster information on their areas of expertise and aggressiveness.

[1135] Step 5:

[1136] The server provides appropriate feedback when a decrease in work efficiency is observed.

[1137] Input: Work assignment list, work efficiency data

[1138] Output: Feedback message

[1139] Specific operation: The server monitors work efficiency data, and if efficiency decreases, it generates a feedback message and sends it to the terminal.

[1140] Step 6:

[1141] The terminal receives feedback and work assignments from the server and notifies the user.

[1142] Input: Feedback message, work assignment list

[1143] Output: User notification

[1144] Specific operation: The terminal displays and notifies the user of the feedback messages and work assignment list received from the server.

[1145] Step 7:

[1146] The user acts based on the notification from the terminal and performs the task.

[1147] Input: User notification

[1148] Output: Work performed

[1149] Specific actions: The user checks the notification from the device and performs the instructed task.

[1150] Example 3

[1151] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1152] In conventional conference systems, participants' opportunities to speak tend to be uneven, and speech is often concentrated on certain participants. Furthermore, due to a lack of feedback on speech, participants may feel reluctant to speak up. Furthermore, when there is silence in the meeting, no supplementary information related to the agenda is provided, which can stall the progress of the meeting. To solve these problems, a system is needed that distributes speaking opportunities equally based on the characteristics of participants, provides positive reactions to speech, and provides supplementary information when there is silence in the meeting.

[1153] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1154] In this invention, the server includes means for collecting participant characteristic data and agenda data, means for analyzing the collected data, means for equally allocating speaking opportunities based on the analysis results, means for acquiring participants' speech content in real time, means for analyzing the acquired speech content and generating positive reactions using a generative AI model, means for monitoring the progress of the meeting in real time, means for searching for supplementary information related to the agenda, and means for providing related supplementary information based on the search results. This makes it possible to equally allocate speaking opportunities based on participants' characteristics, provide positive feedback for speech, and provide supplementary information when the meeting becomes silent.

[1155] "Participant characteristic data" refers to information about meeting participants, such as their areas of expertise, their level of enthusiasm, past comments, and behavioral patterns.

[1156] "Agenda data" refers to information about the themes or topics to be covered at a meeting.

[1157] "Means of collection" refers to the methods and techniques used to obtain participant characteristic data and agenda data.

[1158] "Means of analysis" refers to methods and techniques for analyzing collected data and extracting relevance between participant characteristics and agenda items.

[1159] "Means for allocating speaking opportunities equally" refers to methods and techniques for fairly distributing speaking opportunities, taking into account the characteristics of participants and the number of times they speak.

[1160] "Means of obtaining information in real time" refers to methods and technologies for instantly obtaining the content of participants' comments.

[1161] "Generative AI model" refers to a model that uses artificial intelligence to generate text and comments.

[1162] "Means for generating positive reactions" refers to methods and techniques for generating positive feedback in response to participants' comments.

[1163] "Means for monitoring the progress of a meeting in real time" refers to methods and technologies for instantly monitoring the progress of a meeting.

[1164] "Means for retrieving supplemental information" refers to methods and techniques for retrieving additional information related to the topic.

[1165] "Means for providing supplemental information" refers to methods and techniques for providing participants with the retrieved supplemental information.

[1166] This invention is a system that allocates speaking opportunities equally based on the characteristics of meeting participants, provides positive reactions to comments, and provides supplementary information when the meeting falls silent. Specific embodiments of this system are described below.

[1167] Hardware and software used

[1168] The server uses the following hardware and software:

[1169] Hardware: powerful processor, sufficient memory, storage devices, and network interfaces

[1170] Software: Google Cloud Natural Language API, generative AI models (e.g., OpenAI's GPT-4), Elasticsearch

[1171] Data collection and analysis

[1172] The server collects participant characteristic data and agenda data for the meeting. The participant characteristic data includes areas of expertise, proactiveness, past comments, and behavioral patterns. The agenda data includes information about the themes and topics to be covered in the meeting.

[1173] The server uses the Google Cloud Natural Language API to analyze the collected data, thereby extracting the relevance of participants' characteristics to the agenda.

[1174] Allocation of speaking opportunities

[1175] Based on the analysis results, the server runs an algorithm to distribute speaking opportunities equally. For example, when a topic on which participant A excels comes up, the opportunity to speak is provided to A. Similarly, participant B, who has spoken less frequently, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up.

[1176] Examples:

[1177] If participant A is knowledgeable about "marketing," offer A the opportunity to speak when the topic is "marketing strategy for a new product."

[1178] If participant B is knowledgeable about "technological development" but does not speak often, provide B with an opportunity to speak when the topic is "introduction of new technology."

[1179] Example prompt sentence:

[1180] "Participant A at the meeting is knowledgeable about marketing. If the topic is marketing strategies for a new product, give A a chance to speak."

[1181] Reaction Generation

[1182] The server receives the speeches of the participants in real time and converts them into text using, for example, speech recognition technology.

[1183] The server analyzes the acquired comments and generates positive reactions using a generative AI model (e.g., OpenAI's GPT-4). The generated reactions are fed back to the speaker. For example, a comment such as "That's a fresh perspective" is displayed on the speaker's device.

[1184] Examples:

[1185] If participant C makes a comment about "new product design," the server generates a positive comment such as "That's a fresh perspective" and displays it on C's terminal.

[1186] Example prompt sentence:

[1187] "Participant C made a comment about the design of a new product. Please generate a positive response."

[1188] Providing information

[1189] The server monitors the progress of the conference in real time, for example detecting when no one speaks for a certain period of time.

[1190] The server uses Elasticsearch to find additional information related to the topic, for example, market trends and competitor activity related to "new product development."

[1191] The server provides participants with related supplementary information based on the search results, for example by displaying it on the conference screen.

[1192] Examples:

[1193] If the topic is "new product development" and the meeting falls silent, the server will display information such as "current market trends" and "new product trends of competitors" on the meeting screen.

[1194] Example prompt sentence:

[1195] "The topic is new product development and the meeting is silent. Please provide any relevant supporting information."

[1196] The above is a specific embodiment for carrying out the present invention. This system makes it possible to equally allocate speaking opportunities based on the characteristics of participants, provide positive feedback for comments, and provide supplementary information when the meeting falls silent. The flow of the identification process in the third embodiment will be described with reference to FIG. 15.

[1197] Step 1:

[1198] Data collection

[1199] The server collects meeting participant characteristics and agenda data. As input, it retrieves information such as participants' areas of expertise, proactiveness, past comments, and behavioral patterns from a database. As output, it prepares this data as a dataset for analysis. Specifically, the server sends queries to the database to retrieve the required information.

[1200] Step 2:

[1201] Data analysis

[1202] The server uses the Google Cloud Natural Language API to analyze the collected data. As input, it uses the characteristic data and agenda data collected in step 1. As output, it extracts the relevance between participant characteristics and agendas. Specifically, the server calls the API, analyzes the text data, and generates a relevance score.

[1203] Step 3:

[1204] Allocation of speaking opportunities

[1205] Based on the analysis results, the server runs an algorithm to distribute speaking opportunities equally. As input, it uses the relevance scores obtained in step 2 and the number of times each participant has spoken. As output, it determines the allocation of speaking opportunities to each participant. Specifically, the server runs the algorithm to distribute speaking opportunities equally.

[1206] Step 4:

[1207] Acquiring the content of comments

[1208] The server receives the speeches made by participants in real time. As input, it converts the speeches into text using speech recognition technology. As output, it obtains the textual speeches. Specifically, the server calls a speech recognition API and converts the speech data into text data.

[1209] Step 5:

[1210] Reaction Generation

[1211] The server analyzes the acquired utterances and generates a positive reaction using a generative AI model (e.g., OpenAI's GPT-4). As input, it uses the textual utterances obtained in step 4. As output, it obtains the generated positive reaction. Specifically, the server invokes the generative AI model to generate a positive comment for the utterances.

[1212] Step 6:

[1213] Providing Feedback

[1214] The server feeds back the generated reaction to the speaker. As input, it uses the positive reaction generated in step 5. As output, it displays the reaction on the speaker's device. Specifically, the server sends a message to the speaker's device and displays the reaction.

[1215] Step 7:

[1216] Meeting Monitoring

[1217] The server monitors the progress of the conference in real time. As input, it uses the conference audio data and timestamps of comments. As output, it detects whether the conference is silent or not. Specifically, the server analyzes the audio data and detects when no one has spoken for a certain period of time.

[1218] Step 8:

[1219] Information Search

[1220] The server uses Elasticsearch to search for supplementary information related to the agenda. As input, it uses the silence state detected in step 7 and the agenda data. As output, it obtains the relevant supplementary information. Specifically, the server sends a query to Elasticsearch to search for the relevant information.

[1221] Step 9:

[1222] Providing information

[1223] The server provides participants with relevant supplementary information based on the search results. The supplementary information obtained in step 8 is used as input. The output is to display the supplementary information on the conference screen. Specifically, the server sends a message to the conference screen and displays the supplementary information.

[1224] The above are the specific processing steps of the program of this system.

[1225] (Application example 3)

[1226] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1227] In conventional conferencing and communication systems, participants' opportunities to speak tend to be unevenly distributed, with speech often concentrated on certain participants. Furthermore, a lack of feedback after a comment can cause participants to lose motivation to speak up. Furthermore, when meetings or work stalls, a lack of appropriate supplementary information makes it difficult to make efficient decisions. These problems are particularly pronounced in communication between workers and robots in factories, leading to reduced work efficiency.

[1228] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1229] In this invention, the server inputs characteristics of participants such as their areas of expertise and proactiveness, and includes means for initiating a conversation based on the number of times each participant speaks and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for facilitating communication between workers and robots in a factory, means for equally allocating speaking opportunities, means for providing positive feedback after a speech, and means for providing relevant supplementary information when work stalls. This allows all participants to have an equal opportunity to speak, encourages proactive speaking through feedback after a speech, and provides appropriate supplementary information when the meeting or work stalls, enabling efficient decision-making.

[1230] "Participants" refer to people or robots who take part in a meeting or task.

[1231] "Area of ​​expertise" refers to an area in which a participant has particular knowledge and experience and can demonstrate outstanding abilities.

[1232] "Proactivity" refers to the characteristic of participants acting spontaneously and showing a positive attitude toward speaking and working.

[1233] "Number of remarks" refers to the number of times a participant spoke during a meeting or during work.

[1234] "Methods of initiating conversation" refers to methods of providing participants with opportunities to speak at appropriate times, taking into consideration their characteristics and the number of times they have spoken.

[1235] "Positive reactions" refer to providing positive feedback to participants' comments.

[1236] "Silence" refers to a situation in which no one speaks during a meeting or work session and progress is stagnant.

[1237] "Supplementary Information" refers to additional information related to the agenda or work.

[1238] "Factory workers" refers to human workers in factories.

[1239] A "robot" refers to an automated mechanical device that performs work in a factory.

[1240] "Means for facilitating communication" refers to methods for smoothly exchanging information between workers and robots.

[1241] "Means to distribute speaking opportunities equally" refers to methods that provide all participants with a fair chance to speak.

[1242] "Means of providing positive feedback after a comment" refers to a method of providing a positive response after a participant has commented.

[1243] "Means for providing relevant supplementary information when work stalls" refers to a method for providing useful information to resume work when progress is stalled.

[1244] To implement this invention, it is necessary to build a system that facilitates communication between factory workers and robots. This system inputs the characteristics of participants, such as their areas of expertise and proactiveness, and includes a means for initiating conversations based on the number of times a participant speaks and their characteristics, a means for providing positive reactions after initiating a conversation, and a means for providing supplementary information related to the topic during periods of silence.

[1245] System configuration

[1246] The system consists of the following hardware and software:

[1247] Hardware: Robots in factories, tablets and smartphones used by workers

[1248] Software: A communication system implemented in Python

[1249] Program processing

[1250] The server analyzes participants' past comments and behavioral patterns to input characteristics such as their areas of expertise and proactiveness. This allows it to extract the participants' characteristics. Next, it distributes speaking opportunities equally using a method to allocate speaking opportunities based on the number of comments and participant characteristics. For example, if the topic is "new product development," it provides speaking opportunities to workers who are knowledgeable in that field.

[1251] After a comment is made, positive reactions are generated and feedback is provided to the speaker. This reduces participants' reluctance to speak up. Furthermore, when a meeting or task stalls, supplementary information related to the agenda is provided. For example, information on market trends and competitors' activities regarding "new product development" is provided.

[1252] Specific examples

[1253] For example, the following prompts can be fed to a generative AI model:

[1254] Agenda: New product development

[1255] Worker, please speak about your area of ​​expertise, new product development.

[1256] By inputting this prompt into the generative AI model, it is expected that workers will express their opinions regarding the development of new products. After the comments are made, positive feedback such as "That's a fresh perspective" will be automatically generated.

[1257] In this way, the system ensures that all participants have an equal opportunity to speak, and encourages proactive participation through feedback after speaking. It also provides appropriate supplementary information when meetings or work stalls, enabling efficient decision-making.

[1258] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1259] Step 1:

[1260] The server inputs the participants' characteristics, such as their areas of expertise and proactiveness. Specifically, it analyzes the participants' past comments and behavioral patterns to extract their characteristics. This process takes the participants' comment history data as input and uses a text analysis algorithm to identify their areas of expertise and proactiveness. The output is characteristic data for each participant.

[1261] Step 2:

[1262] The server allocates speaking opportunities equally using a method based on the number of times each participant speaks and the characteristics of each participant. Specifically, it receives the current agenda and participant characteristics as input and provides speaking opportunities to appropriate participants. This process matches the agenda with the participants' areas of expertise and selects the most suitable participant. The output is the name of the selected participant and a speaking prompt.

[1263] Step 3:

[1264] The server sends a speech prompt to the selected participants. Specifically, it sends the generated speech prompt to the participant's terminal. In this process, the server receives the speech prompt as input and sends a message to the participant's terminal. As an output, it generates a speech prompt that is displayed on the participant's terminal.

[1265] Step 4:

[1266] A participant makes a statement. Specifically, the participant inputs the content of the statement through their terminal. In this process, the participant's statement is received as input and sent to the server. The participant's statement data is generated as output.

[1267] Step 5:

[1268] The server generates positive reactions after a comment and provides feedback to the commenter. Specifically, it receives the participant's comment data as input and generates positive feedback using a generative AI model. This process analyzes the content of the comment and generates appropriate positive feedback. A positive feedback message is generated as the output.

[1269] Step 6:

[1270] The server sends the generated positive feedback message to the speaker's terminal. Specifically, the server receives the generated feedback message as input and sends the message to the speaker's terminal. As an output, the server generates a feedback message that is displayed on the speaker's terminal.

[1271] Step 7:

[1272] The server provides supplementary information related to the agenda when a meeting or task stalls. Specifically, it receives the current agenda and related database as input, and searches and extracts relevant supplementary information. This process retrieves information related to the agenda from the database and generates appropriate supplementary information. As an output, a supplementary information message is generated.

[1273] Step 8:

[1274] The server transmits the generated supplemental information message to the participant's terminal. Specifically, the server receives the generated supplemental information message as input and transmits the message to the participant's terminal. As an output, the server generates a supplemental information message that is displayed on the participant's terminal.

[1275] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1276] "Example 1"

[1277] One embodiment of the present invention provides a system that inputs characteristics such as participants' areas of expertise and proactiveness, and includes a means for initiating a conversation based on the number of times participants speak and their characteristics, a means for providing positive reactions after initiating a conversation, a means for providing supplementary information related to the topic during silences, and an emotion engine that recognizes participants' emotions. This system encourages participants to speak and invigorates the meeting. Specifically, it selects the timing and topic for initiating a conversation based on information obtained from the content of participants' comments and behavioral patterns. After a participant speaks, it generates a positive reaction to the comment and provides feedback to the participant. Furthermore, when there is silence in the meeting, it provides supplementary information related to the topic, allowing the meeting to proceed smoothly. The emotion engine that recognizes participants' emotions then grasps the participants' emotional state and responds accordingly.

[1278] "Example 2"

[1279] Another embodiment of the present invention provides a system that inputs characteristics of participants, such as their areas of expertise and proactiveness, and includes a means for initiating a conversation based on the number of times participants speak and their characteristics, a means for providing positive reactions after initiating a conversation, a means for providing supplementary information related to the topic during silences, and an emotion engine for recognizing participants' emotions. This system analyzes participants' past meeting minutes and their social media posts to extract their characteristics. Specifically, when a topic on which Participant A excels comes up, it provides Participant A with an opportunity to speak. Similarly, Participant B, who speaks less frequently, is provided with an opportunity to speak when a topic related to Participant B's area of ​​expertise comes up. Furthermore, it generates positive comments such as "That perspective is fresh" and provides feedback to the participants. If the topic is "new product development," it provides information such as market trends and competitors' activities. The emotion engine that recognizes participants' emotions then grasps their emotional state and responds accordingly.

[1280] "Example 3"

[1281] One embodiment of the present invention provides a system that inputs characteristics such as participants' areas of expertise and proactiveness, and includes a means for initiating a conversation based on the number of times participants speak and their characteristics, a means for providing positive reactions after initiating a conversation, a means for providing supplementary information related to the topic during silences, and an emotion engine that recognizes participants' emotions. This system encourages participants to speak and invigorates the meeting. Specifically, it selects the timing and topic for initiating a conversation based on information obtained from the content of participants' comments and behavioral patterns. After a participant speaks, it generates a positive reaction to the comment and provides feedback to the participant. Furthermore, when there is silence in the meeting, it provides supplementary information related to the topic, allowing the meeting to proceed smoothly. The emotion engine that recognizes participants' emotions then grasps the participants' emotional state and responds accordingly.

[1282] The processing flow of each embodiment will be described below.

[1283] "Example 1"

[1284] Step 1: Input the participants' characteristics, such as their areas of expertise and proactiveness. This is done by analyzing the participants' past statements and behavioral patterns.

[1285] Step 2: Start a conversation based on the number of comments and the characteristics of the participants. Specifically, select the timing and topic to start a conversation based on information obtained from the content of the participants' comments and their behavioral patterns.

[1286] Step 3: React positively after the discussion. This is done by generating a positive reaction to the discussion and providing feedback to the participants.

[1287] Step 4: Provide supplementary information related to the topic during silences. Specifically, when there is silence in the meeting, provide supplementary information related to the topic to keep the meeting running smoothly.

[1288] Step 5: Recognize participants' emotions. Use the emotion engine to understand the participants' emotional state and respond accordingly.

[1289] "Example 2"

[1290] Step 1: Input the participants' characteristics, such as their areas of expertise and proactiveness. This is done by analyzing the participants' past meeting minutes and social media posts to extract their traits.

[1291] Step 2: Initiate the discussion based on the number of times participants speak and the characteristics of the participants. Specifically, when a topic that participant A is good at comes up, provide A with an opportunity to speak. Also, provide participant B, who speaks less, with an opportunity to speak when a topic related to B's area of ​​expertise comes up.

[1292] Step 3: After bringing up the topic, have a positive reaction. Specifically, generate a positive comment such as, "That's a fresh perspective," and provide feedback to the participants.

[1293] Step 4: During silences, provide additional information related to the topic. For example, if the topic is "new product development," provide information on market trends and competitors' activities.

[1294] Step 5: Recognize participants' emotions. Use the emotion engine to understand the participants' emotional state and respond accordingly.

[1295] Example 1

[1296] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1297] Conventional conference systems have difficulty fully understanding the characteristics and emotions of participants and providing appropriate opportunities to speak. Furthermore, they lack sufficient feedback after speaking and supplementary information during silences, making it difficult to liven up the meeting. Furthermore, they are unable to recognize participants' emotional states in real time and respond accordingly, which can lead to a decline in participant satisfaction.

[1298] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1299] In this invention, the server includes means for inputting characteristics of participants, such as their areas of expertise and proactiveness, and for initiating a conversation based on the number of times they speak and their characteristics; means for providing positive reactions after initiating a conversation; means for providing supplementary information related to the topic during silences; means for recognizing participants' emotions; means for analyzing collected data; means for providing speaking opportunities based on the analysis results; means for displaying generated feedback; means for collecting related information; and means for displaying emotional states. This makes it possible to grasp participants' characteristics and emotions and provide appropriate speaking opportunities. Furthermore, providing positive feedback after speaking and supplementary information during silences can help to invigorate the meeting. Furthermore, by recognizing participants' emotional states in real time and responding accordingly, participant satisfaction can be improved.

[1300] "Participant" refers to an individual person taking part in a meeting or discussion.

[1301] "Area of ​​expertise" refers to the area of ​​expertise in which the participant has particular knowledge or skills.

[1302] "Proactivity" refers to participants' willingness and attitude to act voluntarily and express their opinions.

[1303] "Number of remarks" refers to the number of times a participant spoke during a conference.

[1304] "Characteristics" refers to individual attributes of participants, such as their behavioral patterns, personality, and expertise.

[1305] "To raise a topic" means to offer a specific participant an opportunity to speak.

[1306] "Positive reactions" refer to providing positive feedback to participants' comments.

[1307] "Silence" refers to a state in which no one speaks during a meeting.

[1308] "Supplementary Information" refers to additional information or data related to the topic of the conference.

[1309] "Recognizing emotions" refers to judging a participant's emotional state from their facial expressions, tone of voice, etc.

[1310] "Analyzing data" refers to analyzing collected information using statistical or machine learning techniques.

[1311] "Providing an opportunity to speak" means giving participants the opportunity to speak.

[1312] "Displaying feedback" refers to presenting the generated feedback in a visible form to the participants.

[1313] "Collecting relevant information" refers to obtaining information related to the topic of the meeting from the Internet or databases.

[1314] "Displaying an emotional state" refers to visually presenting the perceived emotional state to the participant.

[1315] MODE FOR CARRYING OUT THE INVENTION

[1316] The present invention relates to a system for identifying characteristics of conference participants, providing opportunities to speak, generating positive reactions, and providing supplementary information during silences. Specific embodiments of this system will be described below.

[1317] Data Collection Module

[1318] The server uses a data collection module to understand the characteristics of conference participants. This module inputs characteristics such as participants' areas of expertise and proactiveness. Specifically, it collects participants' profile information and past comment history and stores them in a database. The software used is a database management system (e.g., MySQL).

[1319] Data analysis

[1320] The server uses machine learning algorithms to analyze the collected data, specifically, scikit-learn to model participants' behavioral patterns, and the terminal provides an interface to display the analysis results.

[1321] Message sorting module

[1322] The server uses a speech allocation module based on the analysis results to provide participants with speaking opportunities. This module analyzes the number of times and characteristics of participants' speech and allocates the opportunity to speak at the appropriate time. The terminal displays instructions for speaking. The user speaks according to the instructions.

[1323] Reaction Generation Module

[1324] The server uses a reaction generation module to generate positive reactions after participants make comments. Specifically, it uses natural language generation (NLG) technology (e.g., GPT-3) to generate appropriate feedback. The terminal displays the generated feedback. The user receives the feedback.

[1325] Information Module

[1326] The server uses an information provision module to provide supplementary information when the meeting is silent. This module searches for and provides information related to the meeting topic. Specifically, it uses web scraping technology (e.g., Beautiful Soup) to collect relevant information from the Internet. The terminal displays the collected information. The user confirms the provided information.

[1327] Emotion Engine

[1328] The server uses an emotion engine to recognize participants' emotions. This engine uses emotion recognition algorithms (e.g., OpenCV and deep learning models) to analyze participants' facial expressions and tone of voice. Specifically, it analyzes data collected through cameras and microphones to understand participants' emotional states. The terminal displays the emotional states. Users can check their own emotional states.

[1329] Examples and prompts

[1330] Specific examples

[1331] For example, if participant A refrains from speaking during a meeting, the server uses the speech distribution module to display on the terminal, "Mr. A, what do you think about this?" If participant A then says, "I agree with this idea," the server uses the reaction generation module to generate positive feedback, such as "That's a great idea!", and displays this on the terminal. If the meeting falls silent, the server uses the information provision module to display on the terminal, "There are recent research results related to this topic."

[1332] Prompt Sentence Examples

[1333] "Describe a system that captures meeting participant characteristics, provides opportunities to speak, generates positive reactions, and provides additional information during silences."

[1334] In this way, the roles of the server, terminal, and user are clarified, and the processing of the system is explained in detail.

[1335] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1336] Step 1:

[1337] The server starts the data collection module and displays a participant information input screen on the terminal. The user inputs their areas of expertise and past comment history. The input data is sent to the server and stored in a database. A specific example is when the user inputs, "I'm good at marketing."

[1338] Step 2:

[1339] The server uses a machine learning algorithm to analyze the collected data. Specifically, it uses scikit-learn to model participants' behavioral patterns. The input is the participants' areas of expertise and speech history stored in a database. The output is a classification of the participants' characteristics. The terminal displays the analysis result, saying, "Person A is good at marketing."

[1340] Step 3:

[1341] The server uses a speech distribution module based on the analysis results to display the message "Mr. A, what do you think about this?" on the terminal. The analysis results and the current progress of the meeting are used as input. As output, a speech instruction is generated. The user follows the instruction and says "I agree with this idea."

[1342] Step 4:

[1343] The server uses a reaction generation module to generate a positive reaction after a participant makes a comment. Specifically, it uses GPT-3 to generate feedback such as "That's a great idea!". The participant's comment is used as input. The output is the generated positive feedback. The device displays this feedback, and the user replies, "Thank you."

[1344] Step 5:

[1345] The server uses an information provision module to provide supplementary information when the meeting is silent. Specifically, it uses Beautiful Soup to collect relevant information from the Internet. The current meeting topic is used as input. The relevant supplementary information is generated as output. The terminal displays the collected information as "There are recent research results related to this topic." The user reviews the provided information and comments, "This is useful."

[1346] Step 6:

[1347] The server uses an emotion engine to recognize participants' emotions. Specifically, it uses OpenCV and deep learning models to analyze data collected through cameras and microphones. The inputs are participants' facial expressions and tone of voice. The output is the participant's recognized emotional state. The device displays the emotional state as "Person A is currently excited." The user checks their own emotional state and recognizes that "Yes, I am excited."

[1348] (Application example 1)

[1349] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1350] Conventional meeting support systems and work support systems have difficulty fully understanding the characteristics of participants and workers and providing comments and work instructions at the appropriate time. Furthermore, feedback after comments and work was insufficient, making it difficult to maintain the motivation of participants and workers. Furthermore, there was a lack of means to provide appropriate supplementary information during periods of silence or when work stalls, which led to problems with reduced meeting and work efficiency.

[1351] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1352] In this invention, the server inputs characteristics of participants such as their areas of expertise and proactiveness, and includes means for initiating conversations based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating conversations, means for providing supplementary information related to the topic during silences, means for understanding the characteristics of workers and issuing work instructions at appropriate times, means for providing positive feedback after work is completed, means for providing supplementary information when work stalls, and means for understanding the emotional state of workers and responding accordingly.This makes it possible to improve the efficiency of meetings and work and maintain the motivation of participants and workers.

[1353] "Participant" refers to a person who takes part in a meeting or task.

[1354] "Area of ​​expertise" refers to the areas or skills in which a participant or worker is particularly skilled.

[1355] "Proactivity" refers to the willingness and attitude of participants and workers to act voluntarily.

[1356] "Number of times of speaking" refers to the frequency with which participants speak in a meeting.

[1357] "Characteristics" refers to the individual qualities and behavioral patterns of participants or workers.

[1358] "To raise a topic" means to offer a specific participant an opportunity to speak.

[1359] "Positive reaction" refers to a positive response to the comments or actions of participants or workers.

[1360] "Silence" refers to a temporary stagnation in a meeting or work.

[1361] "Supplementary Information" refers to additional information provided to facilitate a meeting or task.

[1362] "Worker" refers to a person who performs work in a factory or work site.

[1363] "Work instructions" refers to instructions to workers regarding the work content and procedures to be performed.

[1364] "Feedback" refers to evaluations and comments given to workers regarding their actions and results.

[1365] "Emotional state" refers to the current emotional or psychological state of a participant or worker.

[1366] To implement this invention, it is necessary to build a system in which a server, a terminal, and a user work together. A specific embodiment of this system is shown below.

[1367] System Configuration

[1368] The server runs a program that includes the following modules:

[1369] Data collection module: Collects participants' and workers' areas of expertise, proactiveness, past statements, and behavioral patterns.

[1370] Speech allocation module: Provides speaking opportunities at appropriate times based on collected data.

[1371] Reaction generation module: Generates positive reactions after a comment or task is completed.

[1372] Information module: Provides supplementary information during periods of silence or when work is stalled.

[1373] Emotion Engine: Recognizes the emotional state of participants and workers and responds accordingly.

[1374] Hardware and Software

[1375] Hardware: Factory work support robots, cameras, microphones

[1376] software:

[1377] EmotionEngine: An engine for recognizing employee emotions

[1378] PositiveFeedbackGenerator: A module that generates positive feedback

[1379] TaskAllocator: A module that distributes work orders

[1380] SupplementaryInfoProvider: A module that provides supplementary information

[1381] Data processing and calculation

[1382] The server uses a data collection module to collect information on participants' and workers' areas of expertise, proactiveness, past comments, and behavioral patterns. This data is analyzed by a comment allocation module, which provides appropriate opportunities to speak and work instructions at the appropriate time. After a comment is made or a task is completed, a reaction generation module generates positive feedback and provides it to the participant or worker. When there is silence or the work is stagnant, an information provision module provides relevant supplementary information. The emotion engine analyzes data obtained from the camera and microphone, recognizes the emotional state of the participants and workers, and responds accordingly.

[1383] Specific examples

[1384] For example, if you have employees working in a factory, the server might do the following:

[1385] 1. The data collection module collects information about employees' areas of expertise and past work history.

[1386] 2. Based on the data collected by TaskAllocator, work instructions are issued at the appropriate time.

[1387] 3. After the task is completed, the PositiveFeedbackGenerator provides positive feedback.

[1388] 4. When work stalls, SupplementaryInfoProvider provides relevant supplementary information.

[1389] 5. EmotionEngine recognizes employees' emotional state and responds accordingly.

[1390] Prompt Sentence Examples

[1391] "An employee is welding. Consider his past work history and areas of expertise to provide him with the next task instructions. Also, provide positive feedback after completing the task and provide relevant follow-up information when he stalls. Additionally, recognize his emotional state and respond accordingly."

[1392] In this way, the server, terminals, and users work together to improve the efficiency of meetings and work, and it becomes possible to maintain the motivation of participants and workers.

[1393] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1394] Step 1:

[1395] The server uses a data collection module to collect information on participants' and workers' areas of expertise, their proactiveness, past comments, and behavioral patterns. The inputs include video and audio data from cameras and microphones, as well as past work history data. This data is then analyzed to extract the characteristics of participants and workers. The output is feature data.

[1396] Step 2:

[1397] The server uses a speech allocation module to provide speaking opportunities at appropriate times based on the collected feature data. The inputs are feature data and data on the progress of the current meeting or work. The server analyzes this data and determines who should be given the opportunity to speak and when. The output is speech instruction data.

[1398] Step 3:

[1399] The server uses a reaction generation module to generate positive reactions after a comment or task completion. The server uses comment instruction data, comment content data, and task completion data as input. It analyzes these data and generates appropriate positive feedback. Feedback data is obtained as output.

[1400] Step 4:

[1401] The server uses an information provision module to provide supplementary information during periods of silence or stagnation. It uses current meeting and task progress data as input, analyzes this data, and generates relevant supplementary information. The output is the supplementary information data.

[1402] Step 5:

[1403] The server uses an emotion engine to recognize the emotional state of participants and workers and respond accordingly. It uses video and audio data from cameras and microphones as input. It analyzes this data to recognize the emotional state. The output is emotional state data.

[1404] Step 6:

[1405] The terminal displays the utterance instruction data, feedback data, supplementary information data, and emotional state data received from the server and provides them to the user. As input, it uses various data from the server. It displays this data in an appropriate format and provides it to the user. As output, it obtains information displayed in a format that is easy for the user to understand.

[1406] Step 7:

[1407] The user takes appropriate action based on the utterance instructions, feedback, supplementary information, and emotional state displayed on the device. The information displayed on the device is used as input. Based on this information, the next utterance or task is performed. The user's behavior is obtained as output.

[1408] Example 2

[1409] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1410] Conventional conference systems provide speaking opportunities without considering participants' areas of expertise or proactiveness, which often results in biased speech among certain participants. In addition, there is insufficient feedback on participants' comments, making it difficult to stimulate the meeting. Furthermore, there is no way to provide appropriate supplementary information during silences, which can stall the progress of the meeting. To solve these issues, a system is needed that takes into account the characteristics of participants, provides appropriate speaking opportunities, gives positive feedback, and provides necessary supplementary information.

[1411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1412] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, and for initiating a conversation based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for recognizing participants' emotions, means for collecting participants' past comments and behavioral patterns, means for analyzing the collected data and extracting characteristics of participants, means for providing opportunities to speak based on the extracted characteristics, means for generating positive comments using a generative AI model, means for providing supplementary information using an information search engine, and means for understanding participants' emotional states using emotion recognition technology. This makes it possible to provide speaking opportunities, generate positive feedback, and provide appropriate supplementary information taking into account participants' characteristics.

[1413] "Participant" refers to an individual who takes part in a meeting or discussion.

[1414] "Areas of expertise" refers to areas in which participants have particular knowledge and experience and demonstrate outstanding abilities.

[1415] "Proactivity" refers to the attitude and tendency of participants to speak and act proactively in meetings and discussions.

[1416] "Number of comments" refers to the number of times a participant spoke during a meeting or discussion.

[1417] "Positive reactions" refer to providing positive feedback or responses to participants' comments or actions.

[1418] "Supplementary Information" refers to additional information or data relevant to the topic of the meeting or discussion.

[1419] "Emotion recognition" refers to technology that analyzes and identifies participants' emotional states from their facial expressions, voice, etc.

[1420] "Past comments" refers to comments made by participants in previous meetings or discussions.

[1421] "Behavioral patterns" refer to the behavioral tendencies and habits that participants have shown in the past.

[1422] "Data collection" refers to the process of collecting participants' past statements and behavioral patterns.

[1423] "Data analysis" refers to the process of analyzing collected data and extracting meaningful information.

[1424] "Characteristic extraction" refers to the process of identifying participants' characteristics, such as their areas of expertise and proactiveness, based on data analysis.

[1425] "Providing an opportunity to speak" means giving participants an opportunity to speak.

[1426] A "generative AI model" refers to a model that uses artificial intelligence to generate text and comments.

[1427] An "information search engine" refers to a system that searches for information on the Internet and obtains the necessary data.

[1428] "Emotion recognition technology" refers to technology for analyzing and identifying participants' emotional states.

[1429] MODE FOR CARRYING OUT THE INVENTION

[1430] This invention is a system that takes into account the characteristics of participants in meetings and discussions, provides appropriate opportunities to speak, gives positive feedback, and provides necessary supplementary information. Specific embodiments of this system are described below.

[1431] Data collection

[1432] The server collects participants' past comments and behavioral patterns. Specifically, it collects minutes of past meetings and participants' social media posts. This process uses an API to obtain the data. For example, it uses the Twitter API to collect participants' tweets.

[1433] Data analysis and feature extraction

[1434] The server analyzes the collected data and extracts characteristics such as participants' areas of expertise and proactiveness. This process uses natural language processing (NLP) techniques, specifically, analyzing text using the Python libraries NLTK and spaCy.

[1435] Providing opportunities to speak

[1436] The server provides speaking opportunities during the meeting based on the extracted characteristics. For example, when a topic on which participant A excels comes up, the server provides A with a speaking opportunity. Similarly, participant B, who speaks less, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up.

[1437] Generating positive reactions

[1438] The server generates positive reactions to participants' comments. For example, it generates positive comments such as "That's a fresh perspective" and provides feedback to participants. This process uses a generative AI model (e.g., OpenAI's GPT-3).

[1439] Providing supplementary information

[1440] The server provides supplementary information related to the topic. For example, if the topic is "new product development," it provides information on market trends, competitors' activities, etc. This process uses an information search engine or database.

[1441] Emotion Recognition and Response

[1442] The server uses an emotion engine that recognizes participants' emotions to understand their emotional state. For example, if a participant is nervous, it generates a comment to help them relax. This process uses emotion recognition technology (for example, Microsoft Azure's Emotion API).

[1443] Specific examples

[1444] When a user starts a meeting, the server analyzes past meeting minutes and social media posts to extract the characteristics of the participants. For example, if participant A is found to have extensive knowledge of "marketing," the server will provide A with an opportunity to speak when a topic related to "marketing" comes up during the meeting. If participant B does not speak often, the server will provide him with an opportunity to speak when a topic related to B's area of ​​expertise comes up. Furthermore, the server generates positive comments such as "That's a fresh perspective" and provides feedback to the participants. If the topic is "new product development," the server will provide information such as market trends and the actions of competitors. The emotion engine then grasps the participants' emotional state and responds accordingly.

[1445] Prompt Sentence Examples

[1446] "Please explain how to give Participant A an opportunity to speak when a topic that he or she is good at comes up."

[1447] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1448] Step 1: Data collection

[1449] The server collects participants' past comments and behavioral patterns. Specifically, it uses an API to obtain minutes of past meetings and participants' social media posts. For example, it uses the Twitter API to collect participants' tweets. The input is the participants' identification information, and the output is the collected text data.

[1450] Step 2: Data analysis and feature extraction

[1451] The server analyzes the collected data and extracts characteristics such as participants' areas of expertise and proactiveness. This process uses natural language processing (NLP) technology. Specifically, it analyzes text using the Python libraries NLTK and spaCy. The input is the collected text data, and the output is participant characteristic information. The server tokenizes the text and extracts important keywords to identify participants' areas of expertise and proactiveness.

[1452] Step 3: Providing an opportunity to speak

[1453] The server provides speaking opportunities during the meeting based on the extracted characteristics. For example, when a topic that participant A is good at comes up on the agenda, the server provides A with a speaking opportunity. Similarly, participant B, who speaks less, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up on the agenda. The input is the participant's characteristic information and the progress of the meeting, and the output is a message offering a speaking opportunity. The server monitors the progress of the meeting in real time, and when a specific topic comes up on the agenda, it sends a message encouraging participants to speak.

[1454] Step 4: Generate positive reactions

[1455] The server generates positive reactions to participants' comments. For example, it generates positive comments such as "That's a fresh perspective" and provides them as feedback. This process uses a generative AI model (e.g., OpenAI's GPT-3). The input is the participant's comments, and the output is the generated positive comments. The server analyzes the participants' comments and sends prompts to GPT-3 to generate positive comments, which are then sent to the participants.

[1456] Step 5: Provide supporting information

[1457] The server provides supplementary information related to the agenda. For example, if the agenda is "new product development," it provides information such as market trends and competitors' activities. This process uses an information search engine and database. The input is the agenda content, and the output is related supplementary information. The server uses the Google Search API to obtain the latest information related to the agenda and provides it to participants during the meeting.

[1458] Step 6: Recognize and respond to emotions

[1459] The server uses an emotion engine that recognizes participants' emotions to grasp their emotional state. For example, if a participant is nervous, it generates a comment to help them relax. This process uses emotion recognition technology (for example, Microsoft Azure's Emotion API). The input is the participant's facial expressions and voice data, and the output is their emotional state and a corresponding response message. The server uses the Emotion API to identify the participant's emotional state, generates an appropriate response message, and sends it to the participant.

[1460] (Application example 2)

[1461] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1462] Conventional conference systems and work support systems have difficulty in taking into full consideration the characteristics and emotional states of participants and workers, making it difficult to achieve efficient communication and work support. Furthermore, there are problems with biased speaking opportunities and work delays, which can lead to lower overall efficiency. There is a need for a system that can solve these problems and respond appropriately to the characteristics of participants and workers.

[1463] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1464] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, and for initiating conversations based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating conversations, means for providing supplementary information related to the topic during silences, means including an emotion engine for recognizing participants' emotions, means for analyzing worker characteristics and issuing work instructions at appropriate times, means for generating positive comments and providing feedback when workers complete work, means for providing relevant supplementary information when work is delayed, and means for recognizing the emotional state of workers and responding accordingly. This enables appropriate responses according to the characteristics and emotional state of participants and workers, thereby achieving efficient communication and work support.

[1465] "Participant" refers to an individual who participates in a meeting or task.

[1466] "Area of ​​expertise" refers to an area in which a participant or worker has particular knowledge or skills.

[1467] "Proactivity" refers to the willingness and attitude of participants and workers to act voluntarily.

[1468] "Number of comments" refers to the number of times a participant spoke during a meeting or discussion.

[1469] "Participant characteristics" refers to individual characteristics of participants, such as their areas of expertise and proactiveness.

[1470] "Positive reactions" refers to providing positive feedback on the behavior of participants or workers.

[1471] "Silence" refers to a temporary halt in speech or activity during a meeting or task.

[1472] "Supplementary Information" refers to additional information related to an agenda or task.

[1473] An "emotion engine" refers to a system that recognizes the emotional state of participants and workers and responds accordingly.

[1474] "Worker" refers to an individual who performs work in a factory or work site.

[1475] "Work instructions" refers to giving specific work content and procedures to workers.

[1476] "Positive comments" refer to words that express positive evaluation of a worker's behavior or results.

[1477] "When work is delayed" refers to a situation where work is not progressing smoothly and stagnation or delays occur.

[1478] "Emotional state" refers to the current feelings or mood of a participant or worker.

[1479] The system for implementing this invention analyzes the characteristics of participants and workers, issues work instructions at appropriate times, provides positive feedback, and responds according to their emotional state. A specific embodiment of this system is described below.

[1480] Hardware and software used

[1481] Hardware:

[1482] Camera: Used to capture images of workers.

[1483] Robots: Used to provide work instructions and feedback.

[1484] software:

[1485] OpenAI API: Used to generate positive feedback and additional information.

[1486] EmotionRecognizer: Used to recognize the emotional state of workers.

[1487] Data processing and calculation

[1488] The server first analyzes the past comments and behavioral patterns of participants and workers to extract characteristics such as their areas of expertise and proactiveness. This analysis uses natural language processing technology and machine learning algorithms. Specifically, it analyzes meeting minutes and social media posts to extract participants' characteristics.

[1489] The server then issues work instructions at the appropriate time based on the extracted characteristics. For example, when a task that a worker excels at comes up, the server assigns that task to the worker. Also, when the work is stalled, the server provides relevant supplemental information. To achieve this, the server uses a generative AI model to input prompts such as:

[1490] Positive feedback generating prompt:

[1491] Generate positive feedback when workers do a great job.

[1492] Prompt statement for generating additional information:

[1493] Generate supplemental information to provide when work is delayed. Work: Machine maintenance

[1494] Furthermore, the server uses images captured by the camera to recognize the emotional state of the worker. Using EmotionRecognizer, it analyzes the worker's emotions in real time and takes appropriate action, such as encouraging them to take a break, if stress levels are high.

[1495] Specific examples

[1496] For example, imagine a worker performing machine maintenance in a factory. If the worker's past work history is analyzed and it is determined that the worker is skilled at machine operation, the server will instruct the worker to perform machine maintenance work. If the work is progressing smoothly, the server will provide positive feedback such as "Great job!". On the other hand, if the work is delayed, the server will provide supplemental information such as "This part of the maintenance requires special attention."

[1497] In addition, the system analyzes the facial expressions of workers through cameras and responds by telling them to take a short break if they are feeling stressed, thereby improving worker efficiency and satisfaction.

[1498] In this way, it becomes possible to respond appropriately to the characteristics and emotional state of participants and workers, enabling efficient communication and work support.

[1499] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1500] Step 1:

[1501] The server collects the past statements and behavioral patterns of participants and workers. Specifically, it collects data such as meeting minutes and social media posts. The input is data on past statements and behavioral patterns, and the output is the collected data.

[1502] Step 2:

[1503] The server analyzes the collected data and extracts characteristics of participants and workers, such as their areas of expertise and proactiveness. Specifically, it analyzes the data using natural language processing technology and machine learning algorithms. The input is the collected data, and the output is the extracted characteristic information.

[1504] Step 3:

[1505] The server generates work instructions at the appropriate time based on the extracted characteristic information. For example, when a task that a worker excels at arises, the server assigns that task to the worker. The input is the characteristic information and the current work status, and the output is the generated work instructions.

[1506] Step 4:

[1507] The server generates positive feedback when a worker completes a task. Specifically, the generative AI model inputs the following prompt sentence to generate a positive comment. The input is a notification of task completion, and the output is the generated positive comment.

[1508] Generate positive feedback when workers do a great job.

[1509] Step 5:

[1510] The server provides relevant supplemental information when work is delayed. Specifically, the generative AI model generates supplemental information by inputting the following prompt sentence. The input is a notification of work delay, and the output is the generated supplemental information.

[1511] Generate supplemental information to provide when work is delayed. Work: Machine maintenance

[1512] Step 6:

[1513] The server uses images acquired from the camera to recognize the emotional state of the worker. Specifically, it analyzes the images using EmotionRecognizer and recognizes the worker's emotions in real time. The input is the image acquired from the camera, and the output is the recognized emotional state.

[1514] Step 7:

[1515] The server responds according to the recognized emotional state, for example by generating a notification to take a break if stress levels are high. The input is the recognized emotional state, and the output is the generated response notification.

[1516] Example 3

[1517] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1518] In conventional conference systems, participants' opportunities to speak tend to be unevenly distributed, with certain participants often speaking up more than others. Furthermore, feedback on comments is insufficient, leading to participants feeling reluctant to speak up. Furthermore, silence during a meeting can slow down the progress of the meeting. There is a need to provide a conference system that solves these problems and allows all participants to speak up proactively.

[1519] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1520] In this invention, the server inputs characteristics such as participants' areas of expertise and proactiveness, and includes means for initiating a conversation based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for recognizing participants' emotions, and means for responding in accordance with participants' emotions. This allows all participants to have an equal opportunity to speak, reduces reluctance to speak, and makes it possible for the conference to proceed smoothly.

[1521] "Participant's areas of expertise" refer to areas in which individual participants in the conference have particular knowledge and experience and can specialize.

[1522] "Proactivity" refers to the willingness and attitude of participants to speak up and express their opinions voluntarily in a meeting.

[1523] The "number of times a participant speaks" refers to the number of times a participant speaks during a conference.

[1524] "Floor-raising techniques" refer to methods and techniques for providing specific participants with an opportunity to speak during a meeting.

[1525] A "positive reaction" is a reaction that motivates the speaker by providing positive comments or feedback in response to what the participant has said.

[1526] "Supplementary information" refers to additional information or data related to the meeting topic that provides participants with an opportunity to contribute new perspectives.

[1527] "Means of recognizing emotions" refers to techniques and methods for analyzing and understanding participants' emotional states from their facial expressions, voices, etc.

[1528] "Emotionally responsive methods" are methods and techniques for providing appropriate responses or feedback based on the perceived emotional state of participants.

[1529] The present invention is a system for encouraging participants to speak in a conference system and making the conference more lively. This system includes elements of a server, a terminal, and a user, and operates as follows.

[1530] Hardware and software used

[1531] Hardware

[1532] Server: A high-performance computer for running databases and generative AI models.

[1533] Devices: PCs, tablets, smartphones, etc. used by participants

[1534] Network equipment: Routers and switches for connecting servers and devices

[1535] software

[1536] Database management system: Software for storing and managing participant characteristics and meeting agendas

[1537] Generative AI model: An artificial intelligence model for analyzing speech content and generating positive reactions

[1538] Emotion Recognition Engine: Software for analyzing participants' emotional states

[1539] System Operation

[1540] Message sorting module

[1541] The server retrieves participant characteristic data (areas of expertise, number of times each participant speaks, proactiveness, etc.) from a database. Next, it analyzes the meeting agenda and determines which participant's areas of expertise each agenda item relates to. Based on the analysis results, the server distributes speaking opportunities equally among each participant. For example, when a topic that participant A is an expert on comes up, the server provides A with the opportunity to speak.

[1542] Reaction Generation Module

[1543] The server analyzes the content of participants' comments and generates positive reactions based on the content. Using a generative AI model, it generates positive comments in response to the comments and provides feedback to the speaker. For example, it generates a comment such as, "That's a fresh perspective."

[1544] Information Module

[1545] The server detects silence during a meeting. If no one speaks for a certain period of time, it determines that silence has occurred and retrieves supplementary information related to the agenda from a database or external information source. The retrieved supplementary information is provided to participants, giving them an opportunity to speak from a new perspective.

[1546] Emotion Engine

[1547] The server acquires participants' facial expressions and voice data and inputs it into an emotion engine. The emotion engine is used to analyze the participants' emotional state and respond accordingly based on the analysis results. For example, it provides comments to relax participants who are nervous.

[1548] Specific examples

[1549] Suppose the topic of a meeting is "new product development" and participant A is strong in marketing. The server provides A with an opportunity to speak when a topic related to marketing comes up. If participant B is strong in technology development but speaks infrequently, the server will provide B with an opportunity to speak when a topic related to technology development comes up. If silence occurs during a meeting, the server will provide supplementary information such as market trends and the actions of competitors. After a participant speaks, the server generates a positive reaction such as "That's a fresh perspective" and provides feedback to the speaker.

[1550] Prompt Sentence Examples

[1551] "In a meeting about new product development, if participant A is strong in marketing and participant B is strong in technical development but speaks less, explain how you would allocate speaking opportunities, what supplementary information to provide, what positive reactions to generate, and respond according to the emotions of the participants."

[1552] This system allows all participants to have an equal opportunity to speak, reduces reluctance to speak, and makes it possible to smoothly progress the conference. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1553] Step 1:

[1554] The server retrieves participant characteristic data (areas of expertise, number of comments, proactiveness, etc.) from a database. As input, the participant's ID and past comment history are required. The server analyzes this data and identifies each participant's areas of expertise and proactiveness. As output, the characteristic data of each participant is obtained.

[1555] Step 2:

[1556] The server analyzes the meeting agenda. As input, it requires the meeting agenda text. It uses natural language processing techniques to determine which participants' areas of expertise the agenda relates to. As output, it obtains mapping data between the agenda and participants' areas of expertise.

[1557] Step 3:

[1558] Based on the analysis results, the server allocates speaking opportunities equally to each participant. As input, it requires participant characteristic data and mapping data between the agenda and participants' areas of expertise. The server uses this data to execute an algorithm for allocating speaking opportunities. As output, it obtains data on the allocation of speaking opportunities to each participant.

[1559] Step 4:

[1560] The server analyzes the participants' comments. The input requires the participant's comment text. The server uses a generative AI model to analyze the comment and generate a positive reaction based on the content. The output is a positive comment.

[1561] Step 5:

[1562] The server feeds back the generated positive reactions to the speaker. As input, the positive comment and the speaker's ID are required. The server uses this data to send the comment to the speaker. As output, the feedback to the speaker is complete.

[1563] Step 6:

[1564] The server detects silence during a meeting. The input required is the audio data and speech logs of the meeting. The server determines silence when no speech is heard for a certain period of time. The output is the silence detection result.

[1565] Step 7:

[1566] The server retrieves supplementary information related to the agenda. As input, it requires the agenda text and access to external information sources. The server retrieves the supplementary information from a database or external information sources. As output, it obtains the supplementary information data.

[1567] Step 8:

[1568] The server provides the acquired supplementary information to the participants. As input, the supplementary information data and the participant's ID are required. The server uses these data to send the supplementary information to the participants. As output, the provision of information to the participants is complete.

[1569] Step 9:

[1570] The server acquires participants' facial expressions and voice data and inputs it into the emotion engine. The participants' facial expressions and voice data are required as input. The server uses the emotion engine to analyze the participants' emotional states. The emotion analysis results are obtained as output.

[1571] Step 10:

[1572] The server responds based on the participant's emotions based on the analysis results. The inputs required are the emotion analysis results and the participant's ID. The server uses this data to provide appropriate responses and feedback. The output is the completion of the emotional response to the participant.

[1573] (Application example 3)

[1574] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1575] In conventional conference systems, participants' opportunities to speak tend to be unevenly distributed, with certain participants often speaking more than others. Furthermore, feedback after speaking is insufficient, which can lead to participants feeling reluctant to speak up. Furthermore, when there are silences in the meeting, appropriate supplementary information is not provided, which can slow down the progress of the meeting. There is a need to solve these problems and make meetings more lively.

[1576] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1577] In this invention, the server includes means for inputting characteristics such as participants' areas of expertise and proactiveness, and initiating a conversation based on the number of times participants speak and their characteristics, means for providing positive reactions after initiating a conversation, means for providing supplementary information related to the topic during silences, means for recognizing participants' emotions and responding accordingly, means for analyzing the meeting agenda and allocating speaking opportunities based on participants' areas of expertise, means for generating positive comments after participants speak and providing feedback to the speaker, means for providing supplementary information related to the agenda during silences in the meeting, and means for capturing participants' facial expressions and analyzing them with an emotion recognition engine.This encourages participants to speak and makes the meeting more lively.

[1578] "Participant's areas of expertise" are areas in which each participant in the conference has specialized knowledge or experience.

[1579] "Positivity" refers to the willingness and attitude of participants to speak and act voluntarily.

[1580] "Number of remarks" refers to the number of times a participant spoke during a meeting.

[1581] "Participant characteristics" refers to information such as participants' areas of expertise, their proactiveness, past comments, and behavioral patterns.

[1582] "Methods of initiating conversation" are methods of providing participants with opportunities to speak at appropriate times based on their characteristics and the number of times they have spoken.

[1583] "Positive reactions" refer to making positive comments or feedback in response to participants' comments.

[1584] "Supplementary Information" refers to additional information or data related to the agenda.

[1585] "Emotion recognition" refers to analyzing a participant's emotional state from their facial expressions, tone of voice, etc.

[1586] "Analyzing the agenda" refers to understanding the theme or topic of the meeting and extracting information to respond appropriately.

[1587] "Allocating speaking opportunities" refers to adjusting the order and timing of speaking, taking into consideration the participants' areas of expertise and the number of times they will speak.

[1588] "Positive comments" refer to providing positive evaluations or encouraging words in response to participants' comments.

[1589] "Capturing facial expressions" refers to using a camera or other device to capture a participant's facial expressions.

[1590] An "emotion recognition engine" is software that analyzes captured facial expressions and voice data to determine the emotional state of participants.

[1591] The system for carrying out the present invention supports meetings using factory robots. The specific configuration and operation of the system will be described below.

[1592] System configuration

[1593] 1. Hardware:

[1594] Factory robots: Examples include humanoid robots such as Pepper and NAO.

[1595] Microphone and speaker: Used to capture what participants say and provide audio feedback.

[1596] Camera: Used to capture participants' facial expressions and perform emotion recognition.

[1597] 2. Software:

[1598] Speech recognition software: We will use Google Speech-to-Text as an example.

[1599] Emotion Recognition Engine: Using Affectiva as an example.

[1600] Natural language processing engine: Using GPT-4 as an example.

[1601] Database: Used to store participant characteristics and speech history.

[1602] System Operation

[1603] 1. Speech Recognition:

[1604] A microphone captures what participants say, which is then converted into text using speech recognition software, which then stores the speech as digital data.

[1605] 2. Participant characteristics input:

[1606] Participants' areas of expertise and number of comments are stored in a database and loaded at the start of the meeting, allowing the characteristics of each participant to be input into the system.

[1607] 3. Message allocation:

[1608] It uses a natural language processing engine to analyze the agenda and allocate speaking opportunities based on participants' areas of expertise. For example, when a technical topic comes up, it provides speaking opportunities to technical experts.

[1609] 4. Generating positive reactions:

[1610] After a participant speaks, a natural language processing engine is used to generate positive comments and send them through the speaker, helping participants feel less reluctant to speak up.

[1611] 5. Supplementary information provided:

[1612] When a meeting falls silent, the system retrieves supplementary information related to the agenda from a database, formats it appropriately using a natural language processing engine, and provides it, for example, information such as "Current market trends are as follows."

[1613] 6. Emotion recognition:

[1614] The camera captures participants' facial expressions and analyzes them with an emotion recognition engine. Based on the analysis results, appropriate responses are taken. For example, if a participant is nervous, comments to help them relax are generated.

[1615] Examples and prompts

[1616] Specific examples

[1617] Meeting scenario:

[1618] Agenda: New product development

[1619] Participants: Mechanical Engineer A, Marketing Officer B, Quality Control Officer C

[1620] Message sorting:

[1621] When a technical topic that mechanical engineer A is good at comes up, A is given an opportunity to speak.

[1622] Positive reactions:

[1623] After Mechanical Engineer A makes a statement, a comment such as "That's a fresh perspective" is generated.

[1624] Additional information provided:

[1625] When the meeting falls silent, provide information such as, "Here are the current market trends."

[1626] Prompt statement

[1627] "In meetings about new product development, give Mechanical Engineer A the opportunity to speak when technical topics come up and generate positive reactions after he speaks. Also, provide additional information about market trends when the meeting falls silent."

[1628] In this way, factory robots can support meetings, encouraging participants to speak up and making the meeting more lively.

[1629] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1630] Step 1:

[1631] Participant characteristics input

[1632] At the start of a meeting, the server reads the characteristics of participants, such as their areas of expertise and number of comments, from a database. The input includes the participants' past comments and behavioral patterns. Based on this data, the characteristics of the participants are extracted and input into the system. The output is the characteristics of each participant, such as their areas of expertise and proactiveness.

[1633] Step 2:

[1634] Voice Recognition

[1635] The terminal (factory robot) uses a microphone to capture what the participant says. The input contains the participant's voice data. It uses speech recognition software (e.g., Google Speech-to-Text) to convert the voice data into text. The output is the text data of what was said.

[1636] Step 3:

[1637] Speech allocation

[1638] The server analyzes the agenda using a natural language processing engine (e.g., GPT-4). The input includes the text data of the agenda and the characteristics of the participants. Based on the analysis results, it allocates speaking opportunities to participants according to their areas of expertise. The output is a list of participants to whom speaking opportunities will be provided.

[1639] Step 4:

[1640] Generate positive reactions

[1641] The server uses a natural language processing engine to generate positive comments after participants make comments. The input includes text data of the comments. The generated comments are fed back to the speaker via the speaker. The output is text data of the positive comments.

[1642] Step 5:

[1643] Supplementary information provided

[1644] The server retrieves supplementary information related to the agenda from the database when there is silence in the meeting. The input includes the text data of the agenda. The server uses a natural language processing engine to format the supplementary information appropriately and provides it. The output is the text data of the supplementary information.

[1645] Step 6:

[1646] emotion recognition

[1647] The terminal (factory robot) uses a camera to capture the participant's facial expression. The input contains the participant's facial expression data. An emotion recognition engine (e.g., Affectiva) is used to analyze the facial expression data and determine the participant's emotional state. The output is the emotional state data.

[1648] Step 7:

[1649] Responding according to emotions

[1650] The server then responds appropriately based on the analysis results of the emotion recognition engine. The input includes data on the participant's emotional state. For example, if a participant is nervous, the server generates a comment to help them relax. The output is text data for the appropriate response.

[1651] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1652] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1653] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1654] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1655] [Third embodiment]

[1656] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1657] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1658] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1659] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1660] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1661] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1662] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1663] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1664] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1665] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1666] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1667] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1668] "Example 1"

[1669] The present invention is comprised of a data collection module for grasping the characteristics of conference participants, a speech allocation module for providing speaking opportunities, a reaction generation module for generating positive reactions after speaking, and an information provision module for providing supplementary information during silences.

[1670] "Example 2"

[1671] The data collection module analyzes participants' past comments and behavioral patterns to extract characteristics such as their areas of expertise and proactiveness. For example, it is possible to extract participant characteristics by analyzing past meeting minutes and participants' social media posts.

[1672] "Example 3"

[1673] The speech allocation module distributes speaking opportunities equally based on the extracted participant characteristics and the meeting agenda. For example, when a topic on which participant A excels comes up, the opportunity to speak is provided to A. Similarly, participant B, who speaks less frequently, is provided with a speaking opportunity when a topic related to B's area of ​​expertise comes up.

[1674] "Example 4"

[1675] The reaction generation module generates positive reactions and provides feedback to participants after they speak. For example, it generates positive comments such as, "That's a fresh perspective." This helps participants feel less reluctant to speak up.

[1676] "Example 5"

[1677] The information module provides supplementary information related to the agenda when there is silence in the meeting. For example, if the agenda is "new product development," it can provide information on market trends and competitors' activities. This gives participants an opportunity to speak up from a new perspective.

[1678] The processing flow of each embodiment will be described below.

[1679] "Example 1"

[1680] Step 1: The data collection module runs and analyzes participants' past comments and behavioral patterns. This analysis extracts characteristics such as participants' areas of expertise and proactiveness.

[1681] Step 2: The speech allocation module operates and allocates speaking opportunities equally based on the extracted participant characteristics and the meeting agenda.

[1682] Step 3: The reaction generation module operates to generate positive reactions after a participant speaks and provide feedback to the speaker.

[1683] Step 4: The information provision module operates to provide supplementary information related to the agenda when the meeting is silent.

[1684] "Example 2"

[1685] Step 1: The data collection module runs and analyzes participants' past meeting minutes, their social media posts, etc. This analysis extracts the participants' characteristics.

[1686] Step 2: The speech allocation module operates, and when a topic that Participant A is good at comes up, it provides Participant A with an opportunity to speak. Also, Participant B, who has not spoken often, is provided with an opportunity to speak when a topic related to Participant B's area of ​​expertise comes up.

[1687] Step 3: The reaction generation module operates and generates positive comments such as "That's a fresh perspective" and provides feedback to the participants.

[1688] Step 4: The information provision module operates and, if the topic is "new product development," provides information such as market trends and competitors' activities.

[1689] Example 1

[1690] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1691] Conventional conference systems have difficulty fully understanding the characteristics of participants and providing appropriate opportunities to speak. Furthermore, they lack positive reactions after speaking and supplementary information during silences, making it difficult to facilitate smooth progress in meetings or encourage active participation. This has led to problems such as reduced meeting efficiency and productivity.

[1692] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1693] In this invention, the server includes a means for collecting participant profile information and storing it in a database, a means for generating positive reactions using a generative AI model, and a means for searching for and providing related supplemental information using the generative AI model. This makes it possible to understand the characteristics of participants and provide appropriate opportunities to speak. In addition, by providing positive reactions after speaking and supplemental information during silences, the meeting can proceed smoothly and encourage active participation by participants.

[1694] "Participants' specialties, proactiveness, and other characteristics" refers to the characteristics of each person participating in the meeting, such as their expertise and skills, frequency of speaking, and motivation.

[1695] "Number of speeches" refers to the number of times each participant spoke during the conference.

[1696] A "speaking device" refers to a method or system for providing a specific participant with an opportunity to speak during a meeting.

[1697] "Means for positive reactions" refers to methods or systems for showing positive reactions or evaluations after participants make comments.

[1698] "Means for providing supplementary information related to the topic during silence" refers to a method or system for providing additional information related to the topic when silence occurs during a meeting.

[1699] "Profile Information" refers to personal information about you, such as your name, job title, and area of ​​expertise.

[1700] A "database" refers to an information management system that systematically stores collected information and enables it to be searched and used as needed.

[1701] A "generative AI model" refers to a model that has been trained using artificial intelligence techniques to perform a specific task.

[1702] "Supplementary information" refers to additional data or knowledge relevant to the meeting topic.

[1703] "Means for searching and providing" refers to a method or system for searching the Internet or a database for necessary information and providing it to the user.

[1704] MODE FOR CARRYING OUT THE INVENTION

[1705] This invention is a system that understands the characteristics of conference participants, provides appropriate opportunities to speak, generates positive reactions after speaking, and provides supplementary information during silences. This system works in cooperation with three parties: a server, a terminal, and a user.

[1706] Data Collection Module

[1707] 1. A user logs in to a terminal to participate in a conference.

[1708] 2. The device displays a screen for entering the user's profile information.

[1709] 3. The user enters profile information such as name, job title, and area of ​​expertise.

[1710] 4. The terminal sends the entered information to the server.

[1711] 5. The server stores the received profile information in a database.

[1712] Examples:

[1713] The user enters their name and job title into a form displayed on the terminal screen.

[1714] The terminal transmits the input information to the server in real time.

[1715] The server stores the data in the database in the format "User A: Name = Yamada Taro, Position = Manager, Area of ​​Expertise = Marketing."

[1716] Message sorting module

[1717] 1. The server monitors the progress of the conference and schedules speaking opportunities for each participant.

[1718] 2. The server selects appropriate participants for a particular topic based on the collected data.

[1719] 3. The terminal receives instructions from the server and notifies the selected participants when it is time to speak.

[1720] 4. The user receives the notification and makes a comment.

[1721] Examples:

[1722] The server executes logic such as "encourage Yamada Taro, who has expertise in marketing, to speak on agenda item A."

[1723] The device displays the message "Please speak next, Yamada Taro."

[1724] The user confirms the notification and starts speaking.

[1725] Reaction Generation Module

[1726] 1. The server analyzes the user's comments in real time.

[1727] 2. The server inputs a prompt sentence into the generative AI model to generate a positive reaction.

[1728] 3. The device displays the generated reaction to the user.

[1729] 4. The user checks the displayed reaction.

[1730] Examples:

[1731] The server inputs a prompt statement into the generative AI model: "Analyze what User A said and generate a positive reaction."

[1732] The generative AI model generates the reaction, "That's a great idea, Yamada!"

[1733] The device displays this reaction on the screen.

[1734] The user sees the message, "Great idea, Yamada!"

[1735] Information Module

[1736] 1. The server searches for relevant supplemental information during silences in the meeting.

[1737] 2. The server collects relevant data and selects the appropriate information.

[1738] 3. The terminal displays the supplementary information sent from the server to the user.

[1739] 4. The user checks the displayed supplementary information.

[1740] Examples:

[1741] The server inputs the prompt statement "Find the latest news articles related to the current topic" into the generative AI model.

[1742] Generative AI models search for relevant news articles and statistical data and generate information such as, "Here's an article about the latest marketing trends."

[1743] The terminal displays this information on the screen.

[1744] The user sees a message that reads, "Here's an article about the latest marketing trends."

[1745] Prompt Sentence Examples

[1746] "Analyze what User A said and generate a positive reaction."

[1747] "Find the latest news articles related to your current topic."

[1748] The above is an embodiment of the present invention.

[1749] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1750] Step 1:

[1751] A user logs in to a terminal to participate in a conference.

[1752] Input: User login information (user ID, password)

[1753] Output: Login success or failure message

[1754] Specific operation: The user enters their user ID and password into the login screen on the device and clicks the login button. The device sends the entered information to the server, which then performs authentication. If authentication is successful, a login success message is displayed.

[1755] Step 2:

[1756] The terminal displays a screen for inputting user profile information.

[1757] Input: None

[1758] Output: Profile information input screen

[1759] Specific operation: After successfully logging in, the terminal will display a form for the user to enter profile information. The form includes input fields such as name, job title, and area of ​​expertise.

[1760] Step 3:

[1761] Users enter profile information such as name, job title, and area of ​​expertise.

[1762] Input: Profile information such as name, job title, area of ​​expertise, etc.

[1763] Output: Entered profile information

[1764] Specific operation: The user enters information such as name, job title, and area of ​​expertise into the form on the terminal and clicks the submit button.

[1765] Step 4:

[1766] The terminal transmits the input information to the server.

[1767] Input: Profile information entered by the user

[1768] Output: Result sent to the server

[1769] Specific operation: The device sends the profile information entered by the user to the server, which then verifies the received information and stores it in a database.

[1770] Step 5:

[1771] The server stores the received profile information in a database.

[1772] Input: Profile information sent from the device

[1773] Output: Save results to database

[1774] Specific operation: The server saves the received profile information in the database. If the save is successful, it generates a save success message.

[1775] Step 6:

[1776] The server monitors the progress of the conference and schedules speaking opportunities for each participant.

[1777] Input: Meeting progress, participant profile information 【1...

Claims

1. A means for acquiring participant characteristics including areas of expertise and proactiveness of participants of a meeting; A means for providing speaking opportunities to the participants based on the topic of the conference and the participant characteristics; A means for analyzing the content of the speech after providing the speech opportunity and feeding back a positive reaction to the speaker; means for detecting silence during a conference and, upon detection of said silence, providing supplemental information related to the topic to said participants; means for analyzing the emotional state of the participant and taking action according to the emotional state; A system including:

2. The system according to claim 1 , wherein the means for acquiring the participant characteristics of the participant includes means for extracting the participant characteristics by analyzing past utterances and behavioral patterns of the participant.

3. The system according to claim 1 , wherein the means for providing speaking opportunities provides speaking opportunities to the participants in such a way as to equalize the number of times each participant speaks.

Images