System

Abstract

A system is provided.SOLUTION: A system, comprising: means for collecting participant information; means for generating opinions using a generative AI model; means for presenting the generated opinions to participants; means for collecting participant responses and generating additional questions and suggestions; and means for integrating the collected opinions and responses.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many organizations implement group work as part of their human resource training, but the quality of the training depends largely on the participants. In particular, in recent years, remote training has become more common, reducing opportunities for face-to-face interaction, making it difficult to activate group work. This can result in reduced training effectiveness and hinder smooth progress in organizational human resource development. Another problem is that participants are often passive and find it difficult to express diverse opinions, resulting in discussions limited to a limited perspective. The present invention aims to solve these issues, improve the quality of group work, and realize more fulfilling training. [Means for solving the problem]

[0005] The present invention provides a group work support system that utilizes generative AI. This system includes a means for collecting participant information, a means for generating opinions using a generative AI model, a means for presenting the generated opinions to participants, a means for collecting participant responses and generating follow-up questions or suggestions, and a means for integrating and summarizing the collected opinions and responses. This enables diverse opinions and in-depth discussions to be elicited, even in an online environment, and improves the quality of group work. The system is also designed to facilitate more effective discussions by further including a means for analyzing participant information, identifying the interests and tendencies of each participant, and a means for presenting the generated opinions and suggestions to participants in real time.

[0006] "Means for collecting participant information" refers to devices or software for inputting and acquiring user profile information, initial opinions, areas of interest, etc., and storing them in a database.

[0007] "Means for generating opinions using a generative AI model" refers to algorithms and programs that use generative AI to create diverse opinions and suggestions based on collected participant information.

[0008] "Means for presenting generated opinions to participants" refers to the interface or communication means for displaying the opinions and suggestions generated by the generative AI on the user's device and informing participants.

[0009] "Means for collecting participant responses and generating follow-up questions or suggestions" refers to a device or software that receives feedback from participants, analyzes it, and generates further questions or suggestions.

[0010] "Means for synthesizing and summarizing collected opinions and responses" refers to a system or algorithm that aggregates opinions and responses collected from different participants and summarizes the final conclusion or outcome of the discussion.

[0011] "Means for analyzing participant information and identifying each participant's interests and tendencies" refers to software or devices that process collected participant information using data analysis technology to identify each participant's interests and tendencies.

[0012] "Means for presenting generated opinions and suggestions to participants in real time" refers to technologies and devices that allow participants to instantly see the output from the generative AI and enable the exchange of feedback and suggestions on an ongoing basis. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] The present invention is a group work support system that uses generative AI. The purpose of this system is to provide effective human resource training even in a remote environment. The following describes in detail the form for implementing this system.

[0035] This system includes a server, participant terminals, and a generative AI model, and communication between these elements is via a network.

[0036] Program processing

[0037] 1. Start the session

[0038] A user-controlled device sends a command to start a training session to the server, which then launches the generative AI model and completes the initial setup of the session.

[0039] 2. Collection of Participant Information

[0040] After the initial setup is complete, the server displays an input form for profile information and initial opinions on each participant's device. Users enter their own information and initial opinions from their device according to this form and send them to the server. The sent information is then stored in a database on the server.

[0041] 3. Generative AI opinion generation

[0042] The server analyzes the collected participant information to understand each participant's interests and tendencies. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0043] 4. Presenting opinions and deepening discussions

[0044] The generated opinions and suggestions are displayed in real time on each participant's device. Users can then enter their reactions and additional opinions, which are then sent back to the server. The server then receives this information, and the AI ​​analyzes and generates additional questions and suggestions.

[0045] 5. Integration and Conclusion

[0046] Finally, the server integrates all collected opinions and responses and compiles the final conclusion or results of the discussion, which are presented to the participants' devices and, if necessary, stored in a database.

[0047] Specific examples

[0048] For example, consider a case where a company is conducting remote training and group work is conducted on the theme of "improving marketing strategies."

[0049] 1. Session start: The server launches the generative AI model and completes preparations for the session.

[0050] 2. Collecting participant information: Participants input their opinions, such as "I would like to improve the marketing of our company's product A." This is sent to the server.

[0051] 3. Generative AI opinion generation: Generative AI generates opinions that suggest "effective promotion strategies using social media."

[0052] 4. Presenting opinions and deepening the discussion: The suggestions are displayed on participants' devices, and they can input follow-up questions such as, "Which specific platform would be best?" The AI ​​then generates more detailed suggestions, such as, "Instagram and TikTok are suitable for the target demographic."

[0053] 5. Integration and summary of opinions: The server integrates all opinions, compiles a final proposal for improving the marketing strategy, and presents it to the participants.

[0054] The above is an embodiment of the present invention. This system promotes the exchange of diverse opinions and in-depth discussions even in a remote environment, and realizes effective personnel training.

[0055] The processing flow will be explained below.

[0056] Step 1:

[0057] The user (administrator) inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server.

[0058] Step 2:

[0059] The server receives the session start instruction, launches the generative AI model, completes the initial session setup, and sends a session start notification to the participants' devices.

[0060] Step 3:

[0061] Participants fill in the necessary information in the profile information and initial opinion input form displayed on their device. The participant sends this information from their device, and the server stores the received information in a database.

[0062] Step 4:

[0063] The server analyzes the participant information collected to understand each participant's interests and tendencies. Based on the analysis results, a generative AI model is used to generate a variety of opinions and suggestions.

[0064] Step 5:

[0065] The server sends the generated opinions and suggestions to each participant's device in real time, and the participant can check the presented opinions on their device and enter their reactions or additional opinions.

[0066] Step 6:

[0067] Participants' reactions and additional comments are sent from their devices to the server, which receives this information and instructs the generating AI to analyze it.

[0068] Step 7:

[0069] The AI ​​analyzes participants' responses and generates additional questions and suggestions. The generated questions and suggestions are sent to the server, which then sends them to the participants' devices.

[0070] Step 8:

[0071] Participants input their opinions on the additional questions and suggestions presented on their devices and send them to the server, which receives the information again and analyzes and generates it again as necessary.

[0072] Step 9:

[0073] The server finally aggregates all collected opinions and responses, and summarizes the conclusions and results of the discussion. The results are then displayed on the participants' devices.

[0074] Step 10:

[0075] At the end of the session, the server stores all the results in a database, and the terminal displays a "Session has ended" notification to the participant.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] To conduct effective human resource training in a remote environment, participants must be able to exchange opinions and hold discussions smoothly. However, in conventional systems, the processes of collecting, generating, and integrating opinions are often inefficient and lack real-time performance. This limits opportunities for participants to share their opinions and engage in in-depth discussions. Therefore, a system that enables effective and smooth exchange of opinions and discussions, even in a remote environment, is needed.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for collecting participant responses and generating additional questions or suggestions, means for integrating and summarizing the collected opinions and responses, means for activating the generative AI model and initializing the session, means for temporarily storing the generated opinions in the server, and means for presenting the final results to the participants' terminals. This enables efficient and smooth exchange of opinions and discussions even in a remote environment.

[0081] "Participant information" is data including profile information and initial opinions entered by a user.

[0082] A "generative AI model" is an artificial intelligence mechanism that generates diverse opinions and suggestions based on collected participant information.

[0083] "Means for generating opinions" refers to the process and devices that use generative AI models to generate diverse opinions and suggestions.

[0084] The "means for presenting opinions" refers to the process and device for displaying the generated opinions and suggestions on the terminals of the participants.

[0085] The "means for collecting responses" refers to the process and device for collecting responses and additional comments entered by participants in response to generated opinions.

[0086] "Means for generating follow-up questions or suggestions" refers to the process and devices that use a generative AI model to generate new questions or suggestions based on the collected responses.

[0087] A "means for synthesizing opinions and responses" is a process and device for organizing all collected opinions and responses and summarizing the final conclusions and results.

[0088] A "means for initializing a session" is a process and device that performs the necessary settings to begin a training session.

[0089] The "means for temporarily storing opinions on a server" refers to a process and device for storing generated opinions and suggestions on a server for a certain period of time.

[0090] The "means for presenting the final result" refers to the process and device for displaying the integrated final conclusion or the results of the discussion on the terminals of the participants.

[0091] This invention relates to a group work support system using generative AI, and aims to provide effective human resource training in a remote environment. This system includes a server, participant terminals, and a generative AI model, and communication between each element is via a network.

[0092] System Configuration

[0093] This system consists of the following main hardware and software:

[0094] Server: A high-performance computer server that processes data and runs the generative AI model. The main software installed on it is the generative AI model (e.g., GPT-4).

[0095] Participant device: A PC, tablet, or smartphone used by each participant. This device is used to send participant input information to the server and display information sent from the server.

[0096] Network: The network infrastructure for communication between the server and participant devices via the Internet.

[0097] Program processing

[0098] Each process proceeds as follows:

[0099] 1. Start the session

[0100] A command to start a training session is sent from a device managed by the user to the server, which receives the command, launches the generative AI model, and completes the initial setup of the session.

[0101] 2. Collection of Participant Information

[0102] After the initial session setup is complete, the server displays a form for entering profile information and initial opinions on each participant's device. Users enter their own information and initial opinions on their devices and send them to the server. The server stores the received information in a database.

[0103] 3. Generative AI opinion generation

[0104] The server analyzes the collected participant information to understand each participant's interests and tendencies, and uses a generative AI model to generate various opinions and suggestions. The generated opinions are temporarily stored on the server.

[0105] 4. Presenting opinions and deepening discussions

[0106] The server presents the generated opinions and suggestions to each participant's device in real time. Users then input their reactions and additional opinions and send that information back to the server. The server then receives this information and uses the generative AI to generate additional questions and suggestions, which are then presented to the participants.

[0107] 5. Integration and Conclusion

[0108] The server aggregates all collected opinions and responses and compiles the final conclusion or results of the discussion, which are presented to each participant's device and stored in a database if necessary.

[0109] Specific examples

[0110] For example, if a company is conducting remote training and group work on the theme of "improving marketing strategies," it will be used in the following manner.

[0111] 1. Session start: The user sends a session start instruction from the terminal to the server, and the server starts the generation AI and completes the session preparation.

[0112] 2. Collecting participant information: The server displays an information input form on each participant's device, and the user enters their opinion, such as "I would like to improve the marketing of our company's product A," and sends it to the server. The server stores the information in a database.

[0113] 3. Generative AI opinion generation: The server uses a generative AI model based on the collected information to generate opinions such as "effective promotion strategies using social media."

[0114] 4. Presenting opinions and deepening the discussion: The server displays the generated suggestions on each participant's device, and the user inputs additional questions such as "Which specific platform is best?" The AI ​​then generates a suggestion that "Instagram and TikTok are suitable for the target demographic" and presents it again.

[0115] 5. Integration and summary of opinions: The server integrates all opinions, compiles final marketing strategy improvement proposals, and presents them to each participant. The results are stored in a database as needed.

[0116] Prompt Sentence Examples

[0117] "Please use generative AI to provide effective suggestions for improving our marketing strategy. Also, please provide details on the specific platforms you would recommend."

[0118] As described above, this system promotes the exchange of diverse opinions and in-depth discussions even in a remote environment, enabling effective human resource training.

[0119] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0120] Step 1:

[0121] A command to start a training session is sent to the server from a device managed by the user. The input is a "start session" command generated by the user's operation, and the output is the implementation of session initialization on the server side. The server receives this command and launches a generative AI model (e.g., GPT-4). The server performs session initialization, which includes initializing user information and allocating necessary resources. The server logs the completion of the setup and generates a message indicating that it is ready.

[0122] Step 2:

[0123] The server sends a request to each participant's device to display an input form for profile information and initial opinions. The input is the request from the server, and the output is the input form displayed on the device. The device displays the form for the user to input, and the user enters their own information (e.g., name, job title, interests) and initial opinions into the form. When the user clicks the "Submit" button, the input data is sent to the server. The server analyzes the received data and saves it in an SQL database.

[0124] Step 3:

[0125] The server retrieves the collected participant information from the database and analyzes it. The input is the participant information retrieved from the database, and the output is data on the analyzed participants' interests and tendencies. The server provides the analysis results as input to the generative AI model and sends prompts to generate various opinions and suggestions. The generative AI model generates opinions and suggestions and sends them back to the server, which temporarily stores them in cache memory.

[0126] Step 4:

[0127] The server sends the generated opinions and suggestions to each participant's device in real time. The input is opinion data from the generative AI model, and the output is the opinions and suggestions displayed on each participant's device. The device displays this to the user, who then enters their reactions and additional opinions. The information entered by the user is sent to the server. The server analyzes the received reactions and again sends prompts to the generative AI model to generate additional questions or suggestions. The generative AI model generates new suggestions or questions and sends them back to the server. The server again sends this to each participant's device.

[0128] Step 5:

[0129] The server integrates all opinions and responses to compile a final conclusion or discussion result. The input is all collected opinions and responses, and the output is the integrated final result. During the integration process, the server removes duplicate opinions and responses and prioritizes opinions. The resulting final conclusion or proposal is presented to the participants' devices. If necessary, the server stores these results in a database.

[0130] (Application example 1)

[0131] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0132] In order to effectively handle customer service and store operations remotely in a virtual store, a system that can centrally support interactions with diverse customers is required. However, conventional systems have difficulty understanding participants' interests and trends and making appropriate suggestions and responses in real time. They also lack a means to effectively integrate collected opinions and reactions and present optimal responses.

[0133] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0134] In this invention, the server includes a means for collecting participant information, a means for generating opinions using a generative AI model and supporting various interactions in the virtual store, a means for collecting participant responses and generating follow-up questions and suggestions, and a means for presenting the generated opinions to participants in real time and compiling specific countermeasures and suggestions, thereby enabling customer service and store management in the virtual store to be carried out effectively and efficiently.

[0135] "Participant information" refers to data such as the profiles, opinions, and requests of people who use the system, such as customers and staff.

[0136] A "generative AI model" is a model that uses artificial intelligence technology to automatically generate user opinions and suggestions.

[0137] The "opinion generation means" is a function that uses a generative AI model to create opinions and suggestions based on participant information.

[0138] "Presentation means" refers to a method or function for displaying generated opinions and suggestions to the user.

[0139] "Response collection means" is a function for inputting and collecting responses and additional opinions from users.

[0140] "Question generation" is a function that uses a generative AI model to create additional questions and suggestions based on collected responses and data.

[0141] "Integration means" is a function for centralizing and summarizing collected opinions, suggestions, and reactions.

[0142] A "virtual store" is a store that operates on the Internet or in a virtual space and does not have an actual physical location.

[0143] "Interaction support means" is a function that facilitates smooth communication and interaction between customers and staff in a virtual store.

[0144] MODE FOR CARRYING OUT THE INVENTION

[0145] To implement the present invention, the following system configuration and processing method are required.

[0146] System Configuration

[0147] The system of the present invention includes the following elements:

[0148] 1. Hardware

[0149] Server: A central server for running generative AI models and managing the database.

[0150] Device: The device that the user uses as an interface, such as a smartphone, smart glasses, a head-mounted display, or a customer service robot.

[0151] 2. Software

[0152] Generative AI models: Advanced generative AI models such as GPT-4.

[0153] Database System: A database management system such as MySQL.

[0154] Network communication: Communication using HTTP / HTTPS protocols.

[0155] Program processing and data calculation

[0156] 1. Start the session

[0157] The user sends a command to the server to start a session from their device, and the server receives the command, launches the generative AI model, and completes the initial setup.

[0158] 2. Collection of Participant Information

[0159] The server displays a form on each user's device to allow them to enter their profile information and current opinions. The user enters the information and sends it from the device to the server. This information is then stored in a database.

[0160] 3. Opinion generation using generative AI models

[0161] The server analyzes the collected participant information to understand each participant's interests and tendencies. It then uses a generative AI model to generate various opinions and suggestions. The generated opinions are temporarily stored on the server.

[0162] 4. Presenting and discussing opinions

[0163] The generated opinions and suggestions are displayed in real time on the participants' devices. Users can then enter their reactions and additional opinions, which are then sent back to the server. The server then receives this information, and the generative AI model analyzes and generates additional questions and suggestions.

[0164] 5. Integration of opinions and final presentation

[0165] Finally, the server aggregates all collected opinions and responses and compiles final countermeasures and conclusions, which are presented to participants' devices and, if necessary, stored in a database.

[0166] Specific examples

[0167] For example, consider a scenario where a virtual store is promoting a new product, where users provide questions and opinions about the new product, and the generative AI model responds with appropriate suggestions in real time.

[0168] Prompt Sentence Examples

[0169] Start a new product promotion event session. Enter your participant profile information and enter your specific requests and questions. Our AI will then provide you with the best possible suggestions.

[0170] Such a system will enable customer service and store management in virtual stores to be carried out effectively and efficiently.

[0171] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0172] Step 1:

[0173] The user sends an instruction to start a session from their device to the server. The input includes trigger information for starting the session. The server receives this instruction, launches the generative AI model, and completes the initial setup of the session. The launch of the generative AI model is the output.

[0174] Step 2:

[0175] The server displays a form on each user's device for entering profile information and current opinions. The user's profile information and initial opinions are required as input. The user enters the information accordingly and sends it from the device to the server. The server receives this information and stores it in a database. The stored information becomes the output.

[0176] Step 3:

[0177] The server analyzes the collected participant information to understand each participant's interests and tendencies. Participant information stored in a database is required as input. The server uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are output and temporarily stored on the server.

[0178] Step 4:

[0179] The generated opinions and suggestions are presented in real time to the participants' devices. Temporarily saved opinion and suggestion data is required as input. The user then enters their reactions and additional opinions and sends them back to the server. The server receives this information, and the generative AI model analyzes and generates additional questions and suggestions. This is the output.

[0180] Step 5:

[0181] The server integrates all collected opinions and reactions and compiles a final response or conclusion. Resubmitted reactions and additional opinions are required as input. The server integrates the data using a generative AI model to generate a conclusion or response. The generated conclusion is the output, and is displayed on the participants' devices and stored in a database.

[0182] The above is a detailed description of the program's processing steps. In this way, customer service and store operations in the virtual store can be carried out effectively and efficiently.

[0183] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0184] This invention combines a group work support system using generative AI with an emotion engine that recognizes user emotions. The purpose of this system is to provide effective personnel training even in remote environments, and in particular, the emotion engine grasps the emotional state of participants, enhancing the generative AI's ability to generate opinions and progress in discussions.

[0185] System Configuration

[0186] The system includes a server, participant terminals, a generative AI model, and an emotion engine, and communication between these elements is via a network.

[0187] Program processing

[0188] 1. Start the session

[0189] The user (administrator) inputs a command to start a training session into the system. The device receives this input and sends the command to start the session to the server. The server then launches the generative AI model and completes the initial setup of the session.

[0190] 2. Collection of Participant Information

[0191] After the initial setup is complete, the server displays an input form for profile information and initial opinions on each participant's device. Users enter their own information and initial opinions from their device according to this form and send them to the server. The sent information is then stored in a database on the server.

[0192] 3. Collecting Emotional Data

[0193] The device captures participants' emotional data in real time and sends it to a server. For example, it analyzes facial expressions and voice changes using a webcam or microphone. The emotional data is stored on the server and reflected in the generative AI's opinion generation.

[0194] 4. Generative AI opinion generation

[0195] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0196] 5. Presenting opinions and deepening discussions

[0197] The generated opinions and suggestions are displayed in real time on each participant's device. Users then enter their reactions and additional opinions, and this information is sent back to the server. The server receives this information, and the generation AI analyzes and generates additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides the generation AI with appropriate feedback and follow-up questions.

[0198] 6. Integration and Conclusion

[0199] Finally, the server aggregates all collected opinions and responses and summarizes the conclusions and results of the discussion. The results are presented to participants' devices and, if necessary, stored in a database.

[0200] Specific examples

[0201] For example, consider a remote training session in which group work is conducted on the theme of "improving customer service."

[0202] 1. Session start: The server launches the generative AI model and completes the session preparation, including the emotion engine.

[0203] 2. Collecting participant information: Participants input their opinions on how they would like to improve communication with customers. This information is sent to the server.

[0204] 3. Collection of emotional data: Participants' facial expressions and voices are analyzed to collect emotional data such as "current stress level is high."

[0205] 4. Generative AI opinion generation: Based on participants' opinions and emotional data, the generative AI suggests ways to effectively respond to customers while reducing stress.

[0206] 5. Presenting opinions and deepening the discussion: The proposals are displayed on the participants' devices, and they ask, "What are the specific ways to respond?" The emotion engine analyzes the participants' emotional state in response to this question and suggests specific actions to relieve stress.

[0207] 6. Integration and summary of opinions: The server integrates all opinions, compiles a final proposal for improving customer service, and presents it to the participants.

[0208] This completes the implementation of the present invention. This system makes it possible to analyze emotional data using an emotion engine and generate sophisticated opinions using generation AI, enabling effective group work and personnel training even in remote environments.

[0209] The processing flow will be explained below.

[0210] Step 1:

[0211] The user inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server.

[0212] Step 2:

[0213] The server receives the session start instruction, starts the generative AI model and emotion engine, completes the initial session setup, and sends a session start notification to each participant's device.

[0214] Step 3:

[0215] The device displays a form for participants to enter their profile information and initial opinions. Users enter their own information and initial opinions on the device and send them to the server. The sent information is stored in a database.

[0216] Step 4:

[0217] The device collects participants' emotional data in real time. For example, it uses a webcam to recognize facial expressions and a microphone to analyze the tone of voice. The emotional data is then sent to a server.

[0218] Step 5:

[0219] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. Based on the analysis results, the generative AI model generates a variety of opinions and suggestions.

[0220] Step 6:

[0221] The server sends the generated opinions and suggestions to each participant's device in real time, where they are displayed and the user can confirm their content.

[0222] Step 7:

[0223] Users input their reactions to the generated opinions and any additional comments from their devices and send them to the server. The server receives the participants' reaction information, and the generation AI analyzes it again to generate additional questions and suggestions.

[0224] Step 8:

[0225] The device continues to collect the user's emotional data and transmits it to the server, where the emotion engine analyzes it and provides the generative AI with additional suggestions and questions based on the user's emotional state.

[0226] Step 9:

[0227] The server sends new questions and suggestions, including the analysis results of the emotion engine, to participants' devices. Users check these new questions and suggestions on their devices and enter their opinions.

[0228] Step 10:

[0229] The server aggregates all collected opinions and responses and compiles the final conclusion or results of the discussion. This result is sent to the terminal and presented to the user. The server stores the final result in a database.

[0230] The above are the specific processing steps for implementing the present invention. This processing enables emotion analysis using an emotion engine and sophisticated opinion generation using generation AI, enabling effective group work and personnel training even in remote environments.

[0231] Example 2

[0232] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0233] In remote group work and training sessions, communication and exchange of opinions between participants can sometimes be difficult. Therefore, there is a need for a system that can effectively promote interaction in remote environments and grasp and utilize participants' emotions and reactions in real time.

[0234] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0235] In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for collecting participant responses and generating additional questions or suggestions, means for integrating and summarizing the collected opinions and responses, and means for analyzing participant emotion data and reflecting the results in the generative AI model's opinion generation. This allows opinions and suggestions to be generated and presented taking into account the emotions and interests of participants, even in a remote environment, enabling effective discussion and decision-making.

[0236] The "means for collecting participant information" is a function that collects profile information and initial opinions entered by users through their terminals and transmits them to the server.

[0237] "Means for generating opinions using a generative AI model" refers to a function that uses a generative AI model to create diverse opinions and proposals based on collected participant information and emotional data.

[0238] "Means for presenting generated opinions to participants" refers to a function that displays opinions and suggestions created by the generative AI model on each participant's device in real time.

[0239] The "means of collecting participants' responses and generating additional questions or suggestions" is a function that allows participants to input their responses or additional opinions to the generated opinions and send them to the server to generate new questions or suggestions.

[0240] "Means for integrating and summarizing collected opinions and reactions" is a function that organizes and integrates all opinions and reactions collected on the server and creates a final conclusion or proposal.

[0241] The "means of analyzing emotional data and reflecting it in the opinion generation of the generative AI model" is a function that analyzes participants' emotional data obtained using a webcam or microphone and incorporates the results into the opinion generation process of the generative AI model.

[0242] This invention combines a group work support system using a generative AI model with an emotion engine that recognizes the user's emotions. The purpose of this system is to effectively conduct personnel training even in a remote environment, and in particular, the emotion engine grasps the emotional state of participants, enhancing the generative AI's ability to generate opinions and progress in discussions.

[0243] System Configuration

[0244] The system includes a server, participant devices, a generative AI model, and an emotion engine. Communication between these elements is via a network. Specifically, the user devices are PCs or smartphones, and emotion data is collected using webcams and microphones. The server includes a high-performance computing environment, and the generative AI model uses a natural language generation model such as OpenAI's GPT-3. The emotion engine uses libraries such as OpenCV and TensorFlow to analyze changes in facial expressions and voice.

[0245] Program processing

[0246] The system operates in the following steps:

[0247] 1. Start the session

[0248] The user (administrator) inputs a command to start a training session into the system. The device receives this input and sends the command to start the session to the server. The server then launches the generative AI model and completes the initial setup of the session.

[0249] Example prompt: "Please begin remote training."

[0250] 2. Collection of Participant Information

[0251] Once the initial setup is complete, the server displays a form for entering profile information and initial comments on each participant's device. Users enter their information and initial comments on their device and send them to the server. The sent information is stored in the server's database.

[0252] Example prompt: "Please provide your initial opinion on customer service."

[0253] 3. Collecting Emotional Data

[0254] The device captures participants' emotional data in real time and sends it to a server. For example, it analyzes facial expressions and voice changes using a webcam or microphone. The emotional data is stored on the server and reflected in the generative AI's opinion generation.

[0255] Example prompt: "Collect emotion data by analyzing participants' facial expressions."

[0256] 4. Generative AI opinion generation

[0257] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0258] Example prompt: "Generate suggestions for how to improve customer service, including ways to reduce stress."

[0259] 5. Presenting opinions and deepening discussions

[0260] The generated opinions and suggestions are displayed in real time on each participant's device. Users then enter their reactions and additional opinions, and this information is sent back to the server. The server receives this information, and the generation AI analyzes and generates additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides the generation AI with appropriate feedback and follow-up questions.

[0261] Example prompt: "Collect participants' reactions to the suggestions presented and generate follow-up questions or suggestions."

[0262] 6. Integration and Conclusion

[0263] Finally, the server aggregates all collected opinions and responses and summarizes the conclusions and results of the discussion. The results are presented to participants' devices and, if necessary, stored in a database.

[0264] Example prompt: "Synthesize all the feedback and come up with a final proposal for improving customer service."

[0265] Specific examples

[0266] For example, consider a remote training session where group work is conducted on the theme of "improving customer service." Specifically, the user clicks the "Start Session" button on the management screen, and the device notifies the server of this operation. The server then activates the generative AI model and emotion engine, completing the initial setup of the session. A participant enters an opinion, such as "I would like to improve communication with customers," and clicks the send button. The device then sends the information to the server, which then stores it in a database.

[0267] Participants' facial expressions and voices are captured using a webcam and microphone and analyzed in real time. The analysis results are sent to a server, which stores them in a database. The server analyzes the participant's information and emotional data and inputs it into a generative AI model (e.g., GPT-3). The generative AI model suggests "ways to effectively respond to customers while reducing stress." The suggestions are displayed on the participant's device, and the participant asks, "What are the specific ways to respond?" The emotion engine analyzes the participant's emotional state in response to the question and suggests specific actions.

[0268] Finally, the server integrates all opinions, compiles final customer service improvement proposals, and presents them to the participants. If necessary, the proposals can be saved in a database. This system enables effective group work and personnel training, even in remote environments.

[0269] The above is an embodiment of the present invention.

[0270] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0271] Step 1:

[0272] The user (administrator) enters an instruction to start a training session into the system. The user clicks the "Start Session" button on a dedicated management screen on the terminal. This action sends an instruction to start the session from the terminal to the server. The terminal sends this information to a specific API endpoint on the server using an HTTP POST request. The server receives this request and executes the startup script for the generative AI model. The server creates an instance of the generative AI model and performs the necessary initial settings.

[0273] Input: User clicks "Start Session" button

[0274] Output: The server initializes an instance of the generated AI model.

[0275] Step 2:

[0276] After the server has finished launching the generated AI model, it displays an input form for profile information and initial opinions on each participant's device. Specifically, the server sends web page data to each device as an HTTP response, which the device parses and displays to the user. The user enters their own information and initial opinions according to the form and clicks the submit button. The device then sends this information to the server via an HTTP POST request in JSON format. The server establishes a database connection and executes an SQL query to store the received information in the database.

[0277] Input: Profile input form sent from the server, information entered by the user

[0278] Output: User information and initial opinion are saved in the server database.

[0279] Step 3:

[0280] The device acquires participants' emotional data in real time. Using input devices such as a webcam and microphone, the device analyzes changes in facial expressions and voice. This analysis is performed using libraries such as OpenCV and TensorFlow. The analysis results are organized in JSON format and sent to the server using an HTTP POST request. The server stores the received emotional data in a database and converts it into the format required for passing it to the generative AI model.

[0281] Input: Video and audio data acquired through the device's webcam and microphone

[0282] Output: Emotion data sent to the server and its converted format

[0283] Step 4:

[0284] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. The server then preprocesses this data to input it into the generative AI. Preprocessing includes data normalization and emotional state scoring. The server then uses the generative AI model to generate a variety of opinions and suggestions. The server inputs the prompt text and preprocessed data into the generative AI model and receives the model's response. The generated opinions are temporarily stored on the server.

[0285] Input: Collected participant information and emotion data

[0286] Output: Opinions and suggestions generated by the generative AI model

[0287] Step 5:

[0288] The server presents the generated opinions and suggestions to each participant's device in real time. The server sends data to each device using WebSocket or HTTP. The user (participant) enters their reaction or additional opinions, and sends this information from their device to the server. The device receives the user's input and sends it back to the server via an HTTP POST request. The server analyzes this and again uses the generation AI to generate additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides appropriate feedback or follow-up questions to the generation AI.

[0289] Input: Opinions generated by a generative AI model or user responses

[0290] Output: Generates additional questions and suggestions, and feedback based on sentiment data

[0291] Step 6:

[0292] The server aggregates all collected opinions and responses. Specifically, it applies an algorithm to compile the best conclusion or discussion result based on the data collected so far. The resulting results are generated as a web page and sent to each device via an HTTP response. The final results and conclusions are stored in a database for later reference and analysis.

[0293] Input: All collected opinions and reactions

[0294] Output: Final conclusions and recommendations are compiled and presented to each device and stored in a database.

[0295] The above are the specific processing steps and operations of this system.

[0296] (Application example 2)

[0297] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0298] In conventional online group work and remote training, it is difficult to properly grasp participants' emotions, which often leads to inefficient discussions and exchanges of opinions. Furthermore, since emotional data is not collected and analyzed in real time, it is difficult to quickly alleviate participants' stress and frustration. In particular, in virtual stores, where interactions with customers take place in real time, it is necessary to immediately grasp customers' emotions and respond appropriately accordingly, but current technology is not sufficient.

[0299] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for acquiring and analyzing emotional data in real time, means for generating responses based on the emotional data using a generative AI model, means for collecting participant reactions and generating follow-up questions or suggestions, and means for integrating and summarizing the collected opinions and reactions. This makes it possible to grasp the emotional states of participants and customers in real time and generate and present appropriate opinions and suggestions based on that data.

[0300] "Participant information" refers to data such as profile information, initial opinions, interests, and tendencies of individuals who participate in training or discussions.

[0301] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate appropriate opinions or suggestions from input data.

[0302] The "means for generating opinions" is a function that uses a generative AI model to generate diverse opinions and proposals based on collected participant information and emotional data.

[0303] The "means for presenting opinions to participants" is a function for displaying generated opinions and suggestions on participants' terminals in real time.

[0304] "Means for acquiring and analyzing emotional data in real time" refers to a function that uses sensor devices such as cameras and microphones to acquire and analyze participants' emotional states from their facial expressions and voices.

[0305] "Means for generating responses based on emotional data" refers to a function that enables the generative AI model to create appropriate responses or suggestions based on the acquired emotional data.

[0306] "Means for collecting responses and generating additional questions or suggestions" is a function for collecting responses and feedback given by participants to the opinions presented and generating further questions or suggestions based on that.

[0307] "Means for integrating and summarizing collected opinions and responses" is a function for integrating the opinions and responses collected from each participant and summarizing the final conclusion or results of the discussion.

[0308] Basic System Configuration

[0309] This invention is a system that includes a server, a user terminal, a generative AI model, and an emotion engine. Communication between these elements is performed via a network.

[0310] Hardware and Software Configuration

[0311] Hardware: smart glasses or head-mounted display (HMD), webcam, microphone, server

[0312] Software: OpenCV, SpeechRecognition, OpenAI API

[0313] Detailed Description

[0314] 1. Server Role

[0315] The server plays a central role in this system, analyzing data collected from each participant's device, running the generative AI model, analyzing emotional data, and managing the generated opinions and suggestions. Specifically, the following data processing is performed:

[0316] Collection and storage of participant information: Participant profile information and initial opinions are sent from the device to the server and stored in a database.

[0317] Emotional data analysis: Emotional data acquired in real time from webcams and microphones is analyzed to understand participants' emotional states.

[0318] Execution of generative AI model: Based on the collected and analyzed data, opinions and questions are generated using the generative AI model.

[0319] 2. Role of the terminal

[0320] The terminal functions as an interface for participants, and various data is input and output.

[0321] Collecting participant information: Participants enter their profile information and initial opinions on their devices.

[0322] Acquiring emotional data: Using the webcam and microphone built into the device, participants' facial expressions and voice data are acquired in real time.

[0323] Displaying opinions and suggestions: The output results of the generative AI model sent from the server are displayed in real time.

[0324] 3. User Operation

[0325] Users input various information into the system and respond to the generated opinions and suggestions. The response data is then sent back to the server for further data analysis and opinion generation.

[0326] Specific examples

[0327] scenario

[0328] For example, consider a case where a customer asks, "Tell me about a new smartphone" in a virtual store. The system's processing flow in this case is as follows:

[0329] 1. Prompt generation

[0330] In this example, the following prompt is created for the generative AI model:

[0331] "The customer seems excited right now. What the customer said: Tell me about your new smartphone.\n\nSuitable response:"

[0332] 2. Acquisition and analysis of emotion data

[0333] The device's webcam and microphone capture the customer's facial expressions and voice, and the emotion engine analyzes them to determine that the customer is "excited."

[0334] 3. AI-based response generation

[0335] Based on the data collected by the server, the generative AI model generates a response such as:

[0336] "You're excited about the features of your new smartphone! Our latest model, the SmartPhone X, is packed with innovative camera technology and a super-fast processor, delivering a next-generation mobile experience."

[0337] In this way, personalized offers can be made in real time within the virtual store, depending on the customer's emotions.

[0338] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0339] Step 1:

[0340] The user inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server. The input is a command to start the session, and the output is to launch the generative AI model and emotion engine and complete the initial setup of the session.

[0341] Step 2:

[0342] The server displays an input form for profile information and initial opinions on each participant's device. Participants enter their own information and initial opinions from their devices and submit them. The input is the participant's profile information and initial opinions, and the output is that this information is sent to the server and stored in a database.

[0343] Step 3:

[0344] The device acquires participants' emotional data in real time and sends it to the server. The emotional data is acquired using a webcam and microphone installed on the device. The input is the participants' facial expressions and voice data, and the output is data on their emotional state obtained by analyzing this data.

[0345] Step 4:

[0346] The server analyzes the collected participant information and emotional data and uses a generative AI model to generate a variety of opinions and suggestions. Specifically, it generates prompts based on the collected data and inputs them into the generative AI model to generate opinions. The input is participant information and emotional data, and the output is the generated opinions and suggestions.

[0347] Step 5:

[0348] The generated opinions and suggestions are presented to each participant's device in real time. Users can then input their reactions and additional opinions. The input is the participant's reaction to the presented opinions, and the output is the response sent back to the server.

[0349] Step 6:

[0350] The server receives the participant's reaction data, and the emotion engine analyzes it to have the generative AI model generate additional questions or suggestions. A prompt sentence is then generated again and input into the generative AI model to generate new suggestions or questions. The input is the participant's reaction data and emotional state, and the output is a newly generated question or suggestion.

[0351] Step 7:

[0352] The server integrates all collected opinions and reactions and compiles the final conclusion or results of the discussion. This is presented to the participants' devices and stored in a database if necessary. The input is the collected opinion and reaction data, and the output is the integrated conclusion or results of the discussion.

[0353] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0354] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0355] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0356] [Second embodiment]

[0357] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0358] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0359] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0360] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0361] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0362] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0363] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0364] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0365] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0366] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0367] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0368] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0369] The present invention is a group work support system that uses generative AI. The purpose of this system is to provide effective human resource training even in a remote environment. The following describes in detail the form for implementing this system.

[0370] This system includes a server, participant terminals, and a generative AI model, and communication between these elements is via a network.

[0371] Program processing

[0372] 1. Start the session

[0373] A user-controlled device sends a command to start a training session to the server, which then launches the generative AI model and completes the initial setup of the session.

[0374] 2. Collection of Participant Information

[0375] After the initial setup is complete, the server displays an input form for profile information and initial opinions on each participant's device. Users enter their own information and initial opinions from their device according to this form and send them to the server. The sent information is then stored in a database on the server.

[0376] 3. Generative AI opinion generation

[0377] The server analyzes the collected participant information to understand each participant's interests and tendencies. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0378] 4. Presenting opinions and deepening discussions

[0379] The generated opinions and suggestions are displayed in real time on each participant's device. Users can then enter their reactions and additional opinions, which are then sent back to the server. The server then receives this information, and the AI ​​analyzes and generates additional questions and suggestions.

[0380] 5. Integration and Conclusion

[0381] Finally, the server integrates all collected opinions and responses and compiles the final conclusion or results of the discussion, which are presented to the participants' devices and, if necessary, stored in a database.

[0382] Specific examples

[0383] For example, consider a case where a company is conducting remote training and group work is conducted on the theme of "improving marketing strategies."

[0384] 1. Session start: The server launches the generative AI model and completes preparations for the session.

[0385] 2. Collecting participant information: Participants input their opinions, such as "I would like to improve the marketing of our company's product A." This is sent to the server.

[0386] 3. Generative AI opinion generation: Generative AI generates opinions that suggest "effective promotion strategies using social media."

[0387] 4. Presenting opinions and deepening the discussion: The suggestions are displayed on participants' devices, and they can input follow-up questions such as, "Which specific platform would be best?" The AI ​​then generates more detailed suggestions, such as, "Instagram and TikTok are suitable for the target demographic."

[0388] 5. Integration and summary of opinions: The server integrates all opinions, compiles a final proposal for improving the marketing strategy, and presents it to the participants.

[0389] The above is an embodiment of the present invention. This system promotes the exchange of diverse opinions and in-depth discussions even in a remote environment, and realizes effective personnel training.

[0390] The processing flow will be explained below.

[0391] Step 1:

[0392] The user (administrator) inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server.

[0393] Step 2:

[0394] The server receives the session start instruction, launches the generative AI model, completes the initial session setup, and sends a session start notification to the participants' devices.

[0395] Step 3:

[0396] Participants fill in the necessary information in the profile information and initial opinion input form displayed on their device. The participant sends this information from their device, and the server stores the received information in a database.

[0397] Step 4:

[0398] The server analyzes the participant information collected to understand each participant's interests and tendencies. Based on the analysis results, a generative AI model is used to generate a variety of opinions and suggestions.

[0399] Step 5:

[0400] The server sends the generated opinions and suggestions to each participant's device in real time, and the participant can check the presented opinions on their device and enter their reactions or additional opinions.

[0401] Step 6:

[0402] Participants' reactions and additional comments are sent from their devices to the server, which receives this information and instructs the generating AI to analyze it.

[0403] Step 7:

[0404] The AI ​​analyzes participants' responses and generates additional questions and suggestions. The generated questions and suggestions are sent to the server, which then sends them to the participants' devices.

[0405] Step 8:

[0406] Participants input their opinions on the additional questions and suggestions presented on their devices and send them to the server, which receives the information again and analyzes and generates it again as necessary.

[0407] Step 9:

[0408] The server finally aggregates all collected opinions and responses, and summarizes the conclusions and results of the discussion. The results are then displayed on the participants' devices.

[0409] Step 10:

[0410] At the end of the session, the server stores all the results in a database, and the terminal displays a "Session has ended" notification to the participant.

[0411] Example 1

[0412] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0413] To conduct effective human resource training in a remote environment, participants must be able to exchange opinions and hold discussions smoothly. However, in conventional systems, the processes of collecting, generating, and integrating opinions are often inefficient and lack real-time performance. This limits opportunities for participants to share their opinions and engage in in-depth discussions. Therefore, a system that enables effective and smooth exchange of opinions and discussions, even in a remote environment, is needed.

[0414] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0415] In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for collecting participant responses and generating additional questions or suggestions, means for integrating and summarizing the collected opinions and responses, means for activating the generative AI model and initializing the session, means for temporarily storing the generated opinions in the server, and means for presenting the final results to the participants' terminals. This enables efficient and smooth exchange of opinions and discussions even in a remote environment.

[0416] "Participant information" is data including profile information and initial opinions entered by a user.

[0417] A "generative AI model" is an artificial intelligence mechanism that generates diverse opinions and suggestions based on collected participant information.

[0418] "Means for generating opinions" refers to the process and devices that use generative AI models to generate diverse opinions and suggestions.

[0419] The "means for presenting opinions" refers to the process and device for displaying the generated opinions and suggestions on the terminals of the participants.

[0420] The "means for collecting responses" refers to the process and device for collecting responses and additional comments entered by participants in response to generated opinions.

[0421] "Means for generating follow-up questions or suggestions" refers to the process and devices that use a generative AI model to generate new questions or suggestions based on the collected responses.

[0422] A "means for synthesizing opinions and responses" is a process and device for organizing all collected opinions and responses and summarizing the final conclusions and results.

[0423] A "means for initializing a session" is a process and device that performs the necessary settings to begin a training session.

[0424] The "means for temporarily storing opinions on a server" refers to a process and device for storing generated opinions and suggestions on a server for a certain period of time.

[0425] The "means for presenting the final result" refers to the process and device for displaying the integrated final conclusion or the results of the discussion on the terminals of the participants.

[0426] This invention relates to a group work support system using generative AI, and aims to provide effective human resource training in a remote environment. This system includes a server, participant terminals, and a generative AI model, and communication between each element is via a network.

[0427] System Configuration

[0428] This system consists of the following main hardware and software:

[0429] Server: A high-performance computer server that processes data and runs the generative AI model. The main software installed on it is the generative AI model (e.g., GPT-4).

[0430] Participant device: A PC, tablet, or smartphone used by each participant. This device is used to send participant input information to the server and display information sent from the server.

[0431] Network: The network infrastructure for communication between the server and participant devices via the Internet.

[0432] Program processing

[0433] Each process proceeds as follows:

[0434] 1. Start the session

[0435] A command to start a training session is sent from a device managed by the user to the server, which receives the command, launches the generative AI model, and completes the initial setup of the session.

[0436] 2. Collection of Participant Information

[0437] After the initial session setup is complete, the server displays a form for entering profile information and initial opinions on each participant's device. Users enter their own information and initial opinions on their devices and send them to the server. The server stores the received information in a database.

[0438] 3. Generative AI opinion generation

[0439] The server analyzes the collected participant information to understand each participant's interests and tendencies, and uses a generative AI model to generate various opinions and suggestions. The generated opinions are temporarily stored on the server.

[0440] 4. Presenting opinions and deepening discussions

[0441] The server presents the generated opinions and suggestions to each participant's device in real time. Users then input their reactions and additional opinions and send that information back to the server. The server then receives this information and uses the generative AI to generate additional questions and suggestions, which are then presented to the participants.

[0442] 5. Integration and Conclusion

[0443] The server aggregates all collected opinions and responses and compiles the final conclusion or results of the discussion, which are presented to each participant's device and stored in a database if necessary.

[0444] Specific examples

[0445] For example, if a company is conducting remote training and group work on the theme of "improving marketing strategies," it will be used in the following manner.

[0446] 1. Session start: The user sends a session start instruction from the terminal to the server, and the server starts the generation AI and completes the session preparation.

[0447] 2. Collecting participant information: The server displays an information input form on each participant's device, and the user enters their opinion, such as "I would like to improve the marketing of our company's product A," and sends it to the server. The server stores the information in a database.

[0448] 3. Generative AI opinion generation: The server uses a generative AI model based on the collected information to generate opinions such as "effective promotion strategies using social media."

[0449] 4. Presenting opinions and deepening the discussion: The server displays the generated suggestions on each participant's device, and the user inputs additional questions such as "Which specific platform is best?" The AI ​​then generates a suggestion that "Instagram and TikTok are suitable for the target demographic" and presents it again.

[0450] 5. Integration and summary of opinions: The server integrates all opinions, compiles final marketing strategy improvement proposals, and presents them to each participant. The results are stored in a database as needed.

[0451] Prompt Sentence Examples

[0452] "Please use generative AI to provide effective suggestions for improving our marketing strategy. Also, please provide details on the specific platforms you would recommend."

[0453] As described above, this system promotes the exchange of diverse opinions and in-depth discussions even in a remote environment, enabling effective human resource training.

[0454] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0455] Step 1:

[0456] A command to start a training session is sent to the server from a device managed by the user. The input is a "start session" command generated by the user's operation, and the output is the implementation of session initialization on the server side. The server receives this command and launches a generative AI model (e.g., GPT-4). The server performs session initialization, which includes initializing user information and allocating necessary resources. The server logs the completion of the setup and generates a message indicating that it is ready.

[0457] Step 2:

[0458] The server sends a request to each participant's device to display an input form for profile information and initial opinions. The input is the request from the server, and the output is the input form displayed on the device. The device displays the form for the user to input, and the user enters their own information (e.g., name, job title, interests) and initial opinions into the form. When the user clicks the "Submit" button, the input data is sent to the server. The server analyzes the received data and saves it in an SQL database.

[0459] Step 3:

[0460] The server retrieves the collected participant information from the database and analyzes it. The input is the participant information retrieved from the database, and the output is data on the analyzed participants' interests and tendencies. The server provides the analysis results as input to the generative AI model and sends prompts to generate various opinions and suggestions. The generative AI model generates opinions and suggestions and sends them back to the server, which temporarily stores them in cache memory.

[0461] Step 4:

[0462] The server sends the generated opinions and suggestions to each participant's device in real time. The input is opinion data from the generative AI model, and the output is the opinions and suggestions displayed on each participant's device. The device displays this to the user, who then enters their reactions and additional opinions. The information entered by the user is sent to the server. The server analyzes the received reactions and again sends prompts to the generative AI model to generate additional questions or suggestions. The generative AI model generates new suggestions or questions and sends them back to the server. The server again sends this to each participant's device.

[0463] Step 5:

[0464] The server integrates all opinions and responses to compile a final conclusion or discussion result. The input is all collected opinions and responses, and the output is the integrated final result. During the integration process, the server removes duplicate opinions and responses and prioritizes opinions. The resulting final conclusion or proposal is presented to the participants' devices. If necessary, the server stores these results in a database.

[0465] (Application example 1)

[0466] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0467] In order to effectively handle customer service and store operations remotely in a virtual store, a system that can centrally support interactions with diverse customers is required. However, conventional systems have difficulty understanding participants' interests and trends and making appropriate suggestions and responses in real time. They also lack a means to effectively integrate collected opinions and reactions and present optimal responses.

[0468] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0469] In this invention, the server includes a means for collecting participant information, a means for generating opinions using a generative AI model and supporting various interactions in the virtual store, a means for collecting participant responses and generating follow-up questions and suggestions, and a means for presenting the generated opinions to participants in real time and compiling specific countermeasures and suggestions, thereby enabling customer service and store management in the virtual store to be carried out effectively and efficiently.

[0470] "Participant information" refers to data such as the profiles, opinions, and requests of people who use the system, such as customers and staff.

[0471] A "generative AI model" is a model that uses artificial intelligence technology to automatically generate user opinions and suggestions.

[0472] The "opinion generation means" is a function that uses a generative AI model to create opinions and suggestions based on participant information.

[0473] "Presentation means" refers to a method or function for displaying generated opinions and suggestions to the user.

[0474] "Response collection means" is a function for inputting and collecting responses and additional opinions from users.

[0475] "Question generation" is a function that uses a generative AI model to create additional questions and suggestions based on collected responses and data.

[0476] "Integration means" is a function for centralizing and summarizing collected opinions, suggestions, and reactions.

[0477] A "virtual store" is a store that operates on the Internet or in a virtual space and does not have an actual physical location.

[0478] "Interaction support means" is a function that facilitates smooth communication and interaction between customers and staff in a virtual store.

[0479] MODE FOR CARRYING OUT THE INVENTION

[0480] To implement the present invention, the following system configuration and processing method are required.

[0481] System Configuration

[0482] The system of the present invention includes the following elements:

[0483] 1. Hardware

[0484] Server: A central server for running generative AI models and managing the database.

[0485] Device: The device that the user uses as an interface, such as a smartphone, smart glasses, a head-mounted display, or a customer service robot.

[0486] 2. Software

[0487] Generative AI models: Advanced generative AI models such as GPT-4.

[0488] Database System: A database management system such as MySQL.

[0489] Network communication: Communication using HTTP / HTTPS protocols.

[0490] Program processing and data calculation

[0491] 1. Start the session

[0492] The user sends a command to the server to start a session from their device, and the server receives the command, launches the generative AI model, and completes the initial setup.

[0493] 2. Collection of Participant Information

[0494] The server displays a form on each user's device to allow them to enter their profile information and current opinions. The user enters the information and sends it from the device to the server. This information is then stored in a database.

[0495] 3. Opinion generation using generative AI models

[0496] The server analyzes the collected participant information to understand each participant's interests and tendencies. It then uses a generative AI model to generate various opinions and suggestions. The generated opinions are temporarily stored on the server.

[0497] 4. Presenting and discussing opinions

[0498] The generated opinions and suggestions are displayed in real time on the participants' devices. Users can then enter their reactions and additional opinions, which are then sent back to the server. The server then receives this information, and the generative AI model analyzes and generates additional questions and suggestions.

[0499] 5. Integration of opinions and final presentation

[0500] Finally, the server aggregates all collected opinions and responses and compiles final countermeasures and conclusions, which are presented to participants' devices and, if necessary, stored in a database.

[0501] Specific examples

[0502] For example, consider a scenario where a virtual store is promoting a new product, where users provide questions and opinions about the new product, and the generative AI model responds with appropriate suggestions in real time.

[0503] Prompt Sentence Examples

[0504] Start a new product promotion event session. Enter your participant profile information and enter your specific requests and questions. Our AI will then provide you with the best possible suggestions.

[0505] Such a system will enable customer service and store management in virtual stores to be carried out effectively and efficiently.

[0506] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0507] Step 1:

[0508] The user sends an instruction to start a session from their device to the server. The input includes trigger information for starting the session. The server receives this instruction, launches the generative AI model, and completes the initial setup of the session. The launch of the generative AI model is the output.

[0509] Step 2:

[0510] The server displays a form on each user's device for entering profile information and current opinions. The user's profile information and initial opinions are required as input. The user enters the information accordingly and sends it from the device to the server. The server receives this information and stores it in a database. The stored information becomes the output.

[0511] Step 3:

[0512] The server analyzes the collected participant information to understand each participant's interests and tendencies. Participant information stored in a database is required as input. The server uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are output and temporarily stored on the server.

[0513] Step 4:

[0514] The generated opinions and suggestions are presented in real time to the participants' devices. Temporarily saved opinion and suggestion data is required as input. The user then enters their reactions and additional opinions and sends them back to the server. The server receives this information, and the generative AI model analyzes and generates additional questions and suggestions. This is the output.

[0515] Step 5:

[0516] The server integrates all collected opinions and reactions and compiles a final response or conclusion. Resubmitted reactions and additional opinions are required as input. The server integrates the data using a generative AI model to generate a conclusion or response. The generated conclusion is the output, and is displayed on the participants' devices and stored in a database.

[0517] The above is a detailed description of the program's processing steps. In this way, customer service and store operations in the virtual store can be carried out effectively and efficiently.

[0518] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0519] This invention combines a group work support system using generative AI with an emotion engine that recognizes user emotions. The purpose of this system is to provide effective personnel training even in remote environments, and in particular, the emotion engine grasps the emotional state of participants, enhancing the generative AI's ability to generate opinions and progress in discussions.

[0520] System Configuration

[0521] The system includes a server, participant terminals, a generative AI model, and an emotion engine, and communication between these elements is via a network.

[0522] Program processing

[0523] 1. Start the session

[0524] The user (administrator) inputs a command to start a training session into the system. The device receives this input and sends the command to start the session to the server. The server then launches the generative AI model and completes the initial setup of the session.

[0525] 2. Collection of Participant Information

[0526] After the initial setup is complete, the server displays an input form for profile information and initial opinions on each participant's device. Users enter their own information and initial opinions from their device according to this form and send them to the server. The sent information is then stored in a database on the server.

[0527] 3. Collecting Emotional Data

[0528] The device captures participants' emotional data in real time and sends it to a server. For example, it analyzes facial expressions and voice changes using a webcam or microphone. The emotional data is stored on the server and reflected in the generative AI's opinion generation.

[0529] 4. Generative AI opinion generation

[0530] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0531] 5. Presenting opinions and deepening discussions

[0532] The generated opinions and suggestions are displayed in real time on each participant's device. Users then enter their reactions and additional opinions, and this information is sent back to the server. The server receives this information, and the generation AI analyzes and generates additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides the generation AI with appropriate feedback and follow-up questions.

[0533] 6. Integration and Conclusion

[0534] Finally, the server aggregates all collected opinions and responses and summarizes the conclusions and results of the discussion. The results are presented to participants' devices and, if necessary, stored in a database.

[0535] Specific examples

[0536] For example, consider a remote training session in which group work is conducted on the theme of "improving customer service."

[0537] 1. Session start: The server launches the generative AI model and completes the session preparation, including the emotion engine.

[0538] 2. Collecting participant information: Participants input their opinions on how they would like to improve communication with customers. This information is sent to the server.

[0539] 3. Collection of emotional data: Participants' facial expressions and voices are analyzed to collect emotional data such as "current stress level is high."

[0540] 4. Generative AI opinion generation: Based on participants' opinions and emotional data, the generative AI suggests ways to effectively respond to customers while reducing stress.

[0541] 5. Presenting opinions and deepening the discussion: The proposals are displayed on the participants' devices, and they ask, "What are the specific ways to respond?" The emotion engine analyzes the participants' emotional state in response to this question and suggests specific actions to relieve stress.

[0542] 6. Integration and summary of opinions: The server integrates all opinions, compiles a final proposal for improving customer service, and presents it to the participants.

[0543] This completes the implementation of the present invention. This system makes it possible to analyze emotional data using an emotion engine and generate sophisticated opinions using generation AI, enabling effective group work and personnel training even in remote environments.

[0544] The processing flow will be explained below.

[0545] Step 1:

[0546] The user inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server.

[0547] Step 2:

[0548] The server receives the session start instruction, starts the generative AI model and emotion engine, completes the initial session setup, and sends a session start notification to each participant's device.

[0549] Step 3:

[0550] The device displays a form for participants to enter their profile information and initial opinions. Users enter their own information and initial opinions on the device and send them to the server. The sent information is stored in a database.

[0551] Step 4:

[0552] The device collects participants' emotional data in real time. For example, it uses a webcam to recognize facial expressions and a microphone to analyze the tone of voice. The emotional data is then sent to a server.

[0553] Step 5:

[0554] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. Based on the analysis results, the generative AI model generates a variety of opinions and suggestions.

[0555] Step 6:

[0556] The server sends the generated opinions and suggestions to each participant's device in real time, where they are displayed and the user can confirm their content.

[0557] Step 7:

[0558] Users input their reactions to the generated opinions and any additional comments from their devices and send them to the server. The server receives the participants' reaction information, and the generation AI analyzes it again to generate additional questions and suggestions.

[0559] Step 8:

[0560] The device continues to collect the user's emotional data and transmits it to the server, where the emotion engine analyzes it and provides the generative AI with additional suggestions and questions based on the user's emotional state.

[0561] Step 9:

[0562] The server sends new questions and suggestions, including the analysis results of the emotion engine, to participants' devices. Users check these new questions and suggestions on their devices and enter their opinions.

[0563] Step 10:

[0564] The server aggregates all collected opinions and responses and compiles the final conclusion or results of the discussion. This result is sent to the terminal and presented to the user. The server stores the final result in a database.

[0565] The above are the specific processing steps for implementing the present invention. This processing enables emotion analysis using an emotion engine and sophisticated opinion generation using generation AI, enabling effective group work and personnel training even in remote environments.

[0566] Example 2

[0567] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0568] In remote group work and training sessions, communication and exchange of opinions between participants can sometimes be difficult. Therefore, there is a need for a system that can effectively promote interaction in remote environments and grasp and utilize participants' emotions and reactions in real time.

[0569] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0570] In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for collecting participant responses and generating additional questions or suggestions, means for integrating and summarizing the collected opinions and responses, and means for analyzing participant emotion data and reflecting the results in the generative AI model's opinion generation. This allows opinions and suggestions to be generated and presented taking into account the emotions and interests of participants, even in a remote environment, enabling effective discussion and decision-making.

[0571] The "means for collecting participant information" is a function that collects profile information and initial opinions entered by users through their terminals and transmits them to the server.

[0572] "Means for generating opinions using a generative AI model" refers to a function that uses a generative AI model to create diverse opinions and proposals based on collected participant information and emotional data.

[0573] "Means for presenting generated opinions to participants" refers to a function that displays opinions and suggestions created by the generative AI model on each participant's device in real time.

[0574] The "means of collecting participants' responses and generating additional questions or suggestions" is a function that allows participants to input their responses or additional opinions to the generated opinions and send them to the server to generate new questions or suggestions.

[0575] "Means for integrating and summarizing collected opinions and reactions" is a function that organizes and integrates all opinions and reactions collected on the server and creates a final conclusion or proposal.

[0576] The "means of analyzing emotional data and reflecting it in the opinion generation of the generative AI model" is a function that analyzes participants' emotional data obtained using a webcam or microphone and incorporates the results into the opinion generation process of the generative AI model.

[0577] This invention combines a group work support system using a generative AI model with an emotion engine that recognizes the user's emotions. The purpose of this system is to effectively conduct personnel training even in a remote environment, and in particular, the emotion engine grasps the emotional state of participants, enhancing the generative AI's ability to generate opinions and progress in discussions.

[0578] System Configuration

[0579] The system includes a server, participant devices, a generative AI model, and an emotion engine. Communication between these elements is via a network. Specifically, the user devices are PCs or smartphones, and emotion data is collected using webcams and microphones. The server includes a high-performance computing environment, and the generative AI model uses a natural language generation model such as OpenAI's GPT-3. The emotion engine uses libraries such as OpenCV and TensorFlow to analyze changes in facial expressions and voice.

[0580] Program processing

[0581] The system operates in the following steps:

[0582] 1. Start the session

[0583] The user (administrator) inputs a command to start a training session into the system. The device receives this input and sends the command to start the session to the server. The server then launches the generative AI model and completes the initial setup of the session.

[0584] Example prompt: "Please begin remote training."

[0585] 2. Collection of Participant Information

[0586] Once the initial setup is complete, the server displays a form for entering profile information and initial comments on each participant's device. Users enter their information and initial comments on their device and send them to the server. The sent information is stored in the server's database.

[0587] Example prompt: "Please provide your initial opinion on customer service."

[0588] 3. Collecting Emotional Data

[0589] The device captures participants' emotional data in real time and sends it to a server. For example, it analyzes facial expressions and voice changes using a webcam or microphone. The emotional data is stored on the server and reflected in the generative AI's opinion generation.

[0590] Example prompt: "Collect emotion data by analyzing participants' facial expressions."

[0591] 4. Generative AI opinion generation

[0592] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0593] Example prompt: "Generate suggestions for how to improve customer service, including ways to reduce stress."

[0594] 5. Presenting opinions and deepening discussions

[0595] The generated opinions and suggestions are displayed in real time on each participant's device. Users then enter their reactions and additional opinions, and this information is sent back to the server. The server receives this information, and the generation AI analyzes and generates additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides the generation AI with appropriate feedback and follow-up questions.

[0596] Example prompt: "Collect participants' reactions to the suggestions presented and generate follow-up questions or suggestions."

[0597] 6. Integration and Conclusion

[0598] Finally, the server aggregates all collected opinions and responses and summarizes the conclusions and results of the discussion. The results are presented to participants' devices and, if necessary, stored in a database.

[0599] Example prompt: "Synthesize all the feedback and come up with a final proposal for improving customer service."

[0600] Specific examples

[0601] For example, consider a remote training session where group work is conducted on the theme of "improving customer service." Specifically, the user clicks the "Start Session" button on the management screen, and the device notifies the server of this operation. The server then activates the generative AI model and emotion engine, completing the initial setup of the session. A participant enters an opinion, such as "I would like to improve communication with customers," and clicks the send button. The device then sends the information to the server, which then stores it in a database.

[0602] Participants' facial expressions and voices are captured using a webcam and microphone and analyzed in real time. The analysis results are sent to a server, which stores them in a database. The server analyzes the participant's information and emotional data and inputs it into a generative AI model (e.g., GPT-3). The generative AI model suggests "ways to effectively respond to customers while reducing stress." The suggestions are displayed on the participant's device, and the participant asks, "What are the specific ways to respond?" The emotion engine analyzes the participant's emotional state in response to the question and suggests specific actions.

[0603] Finally, the server integrates all opinions, compiles final customer service improvement proposals, and presents them to the participants. If necessary, the proposals can be saved in a database. This system enables effective group work and personnel training, even in remote environments.

[0604] The above is an embodiment of the present invention.

[0605] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0606] Step 1:

[0607] The user (administrator) enters an instruction to start a training session into the system. The user clicks the "Start Session" button on a dedicated management screen on the terminal. This action sends an instruction to start the session from the terminal to the server. The terminal sends this information to a specific API endpoint on the server using an HTTP POST request. The server receives this request and executes the startup script for the generative AI model. The server creates an instance of the generative AI model and performs the necessary initial settings.

[0608] Input: User clicks "Start Session" button

[0609] Output: The server initializes an instance of the generated AI model.

[0610] Step 2:

[0611] After the server has finished launching the generated AI model, it displays an input form for profile information and initial opinions on each participant's device. Specifically, the server sends web page data to each device as an HTTP response, which the device parses and displays to the user. The user enters their own information and initial opinions according to the form and clicks the submit button. The device then sends this information to the server via an HTTP POST request in JSON format. The server establishes a database connection and executes an SQL query to store the received information in the database.

[0612] Input: Profile input form sent from the server, information entered by the user

[0613] Output: User information and initial opinion are saved in the server database.

[0614] Step 3:

[0615] The device acquires participants' emotional data in real time. Using input devices such as a webcam and microphone, the device analyzes changes in facial expressions and voice. This analysis is performed using libraries such as OpenCV and TensorFlow. The analysis results are organized in JSON format and sent to the server using an HTTP POST request. The server stores the received emotional data in a database and converts it into the format required for passing it to the generative AI model.

[0616] Input: Video and audio data acquired through the device's webcam and microphone

[0617] Output: Emotion data sent to the server and its converted format

[0618] Step 4:

[0619] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. The server then preprocesses this data to input it into the generative AI. Preprocessing includes data normalization and emotional state scoring. The server then uses the generative AI model to generate a variety of opinions and suggestions. The server inputs the prompt text and preprocessed data into the generative AI model and receives the model's response. The generated opinions are temporarily stored on the server.

[0620] Input: Collected participant information and emotion data

[0621] Output: Opinions and suggestions generated by the generative AI model

[0622] Step 5:

[0623] The server presents the generated opinions and suggestions to each participant's device in real time. The server sends data to each device using WebSocket or HTTP. The user (participant) enters their reaction or additional opinions, and sends this information from their device to the server. The device receives the user's input and sends it back to the server via an HTTP POST request. The server analyzes this and again uses the generation AI to generate additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides appropriate feedback or follow-up questions to the generation AI.

[0624] Input: Opinions generated by a generative AI model or user responses

[0625] Output: Generates additional questions and suggestions, and feedback based on sentiment data

[0626] Step 6:

[0627] The server aggregates all collected opinions and responses. Specifically, it applies an algorithm to compile the best conclusion or discussion result based on the data collected so far. The resulting results are generated as a web page and sent to each device via an HTTP response. The final results and conclusions are stored in a database for later reference and analysis.

[0628] Input: All collected opinions and reactions

[0629] Output: Final conclusions and recommendations are compiled and presented to each device and stored in a database.

[0630] The above are the specific processing steps and operations of this system.

[0631] (Application example 2)

[0632] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0633] In conventional online group work and remote training, it is difficult to properly grasp participants' emotions, which often leads to inefficient discussions and exchanges of opinions. Furthermore, since emotional data is not collected and analyzed in real time, it is difficult to quickly alleviate participants' stress and frustration. In particular, in virtual stores, where interactions with customers take place in real time, it is necessary to immediately grasp customers' emotions and respond appropriately accordingly, but current technology is not sufficient.

[0634] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for acquiring and analyzing emotional data in real time, means for generating responses based on the emotional data using a generative AI model, means for collecting participant reactions and generating follow-up questions or suggestions, and means for integrating and summarizing the collected opinions and reactions. This makes it possible to grasp the emotional states of participants and customers in real time and generate and present appropriate opinions and suggestions based on that data.

[0635] "Participant information" refers to data such as profile information, initial opinions, interests, and tendencies of individuals who participate in training or discussions.

[0636] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate appropriate opinions or suggestions from input data.

[0637] The "means for generating opinions" is a function that uses a generative AI model to generate diverse opinions and proposals based on collected participant information and emotional data.

[0638] The "means for presenting opinions to participants" is a function for displaying generated opinions and suggestions on participants' terminals in real time.

[0639] "Means for acquiring and analyzing emotional data in real time" refers to a function that uses sensor devices such as cameras and microphones to acquire and analyze participants' emotional states from their facial expressions and voices.

[0640] "Means for generating responses based on emotional data" refers to a function that enables the generative AI model to create appropriate responses or suggestions based on the acquired emotional data.

[0641] "Means for collecting responses and generating additional questions or suggestions" is a function for collecting responses and feedback given by participants to the opinions presented and generating further questions or suggestions based on that.

[0642] "Means for integrating and summarizing collected opinions and responses" is a function for integrating the opinions and responses collected from each participant and summarizing the final conclusion or results of the discussion.

[0643] Basic System Configuration

[0644] This invention is a system that includes a server, a user terminal, a generative AI model, and an emotion engine. Communication between these elements is performed via a network.

[0645] Hardware and Software Configuration

[0646] Hardware: smart glasses or head-mounted display (HMD), webcam, microphone, server

[0647] Software: OpenCV, SpeechRecognition, OpenAI API

[0648] Detailed Description

[0649] 1. Server Role

[0650] The server plays a central role in this system, analyzing data collected from each participant's device, running the generative AI model, analyzing emotional data, and managing the generated opinions and suggestions. Specifically, the following data processing is performed:

[0651] Collection and storage of participant information: Participant profile information and initial opinions are sent from the device to the server and stored in a database.

[0652] Emotional data analysis: Emotional data acquired in real time from webcams and microphones is analyzed to understand participants' emotional states.

[0653] Execution of generative AI model: Based on the collected and analyzed data, opinions and questions are generated using the generative AI model.

[0654] 2. Role of the terminal

[0655] The terminal functions as an interface for participants, and various data is input and output.

[0656] Collecting participant information: Participants enter their profile information and initial opinions on their devices.

[0657] Acquiring emotional data: Using the webcam and microphone built into the device, participants' facial expressions and voice data are acquired in real time.

[0658] Displaying opinions and suggestions: The output results of the generative AI model sent from the server are displayed in real time.

[0659] 3. User Operation

[0660] Users input various information into the system and respond to the generated opinions and suggestions. The response data is then sent back to the server for further data analysis and opinion generation.

[0661] Specific examples

[0662] scenario

[0663] For example, consider a case where a customer asks, "Tell me about a new smartphone" in a virtual store. The system's processing flow in this case is as follows:

[0664] 1. Prompt generation

[0665] In this example, the following prompt is created for the generative AI model:

[0666] "The customer seems excited right now. What the customer said: Tell me about your new smartphone.\n\nSuitable response:"

[0667] 2. Acquisition and analysis of emotion data

[0668] The device's webcam and microphone capture the customer's facial expressions and voice, and the emotion engine analyzes them to determine that the customer is "excited."

[0669] 3. AI-based response generation

[0670] Based on the data collected by the server, the generative AI model generates a response such as:

[0671] "You're excited about the features of your new smartphone! Our latest model, the SmartPhone X, is packed with innovative camera technology and a super-fast processor, delivering a next-generation mobile experience."

[0672] In this way, personalized offers can be made in real time within the virtual store, depending on the customer's emotions.

[0673] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0674] Step 1:

[0675] The user inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server. The input is a command to start the session, and the output is to launch the generative AI model and emotion engine and complete the initial setup of the session.

[0676] Step 2:

[0677] The server displays an input form for profile information and initial opinions on each participant's device. Participants enter their own information and initial opinions from their devices and submit them. The input is the participant's profile information and initial opinions, and the output is that this information is sent to the server and stored in a database.

[0678] Step 3:

[0679] The device acquires participants' emotional data in real time and sends it to the server. The emotional data is acquired using a webcam and microphone installed on the device. The input is the participants' facial expressions and voice data, and the output is data on their emotional state obtained by analyzing this data.

[0680] Step 4:

[0681] The server analyzes the collected participant information and emotional data and uses a generative AI model to generate a variety of opinions and suggestions. Specifically, it generates prompts based on the collected data and inputs them into the generative AI model to generate opinions. The input is participant information and emotional data, and the output is the generated opinions and suggestions.

[0682] Step 5:

[0683] The generated opinions and suggestions are presented to each participant's device in real time. Users can then input their reactions and additional opinions. The input is the participant's reaction to the presented opinions, and the output is the response sent back to the server.

[0684] Step 6:

[0685] The server receives the participant's reaction data, and the emotion engine analyzes it to have the generative AI model generate additional questions or suggestions. A prompt sentence is then generated again and input into the generative AI model to generate new suggestions or questions. The input is the participant's reaction data and emotional state, and the output is a newly generated question or suggestion.

[0686] Step 7:

[0687] The server integrates all collected opinions and reactions and compiles the final conclusion or results of the discussion. This is presented to the participants' devices and stored in a database if necessary. The input is the collected opinion and reaction data, and the output is the integrated conclusion or results of the discussion.

[0688] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0689] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0690] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0691] [Third embodiment]

[0692] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0693] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0694] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0695] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0696] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0697] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0698] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0699] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0700] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0701] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0702] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0703] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0704] The present invention is a group work support system that uses generative AI. The purpose of this system is to provide effective human resource training even in a remote environment. The following describes in detail the form for implementing this system.

[0705] This system includes a server, participant terminals, and a generative AI model, and communication between these elements is via a network.

[0706] Program processing

[0707] 1. Start the session

[0708] A user-controlled device sends a command to start a training session to the server, which then launches the generative AI model and completes the initial setup of the session.

[0709] 2. Collection of Participant Information

[0710] After the initial setup is complete, the server displays an input form for profile information and initial opinions on each participant's device. Users enter their own information and initial opinions from their device according to this form and send them to the server. The sent information is then stored in a database on the server.

[0711] 3. Generative AI opinion generation

[0712] The server analyzes the collected participant information to understand each participant's interests and tendencies. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0713] 4. Presenting opinions and deepening discussions

[0714] The generated opinions and suggestions are displayed in real time on each participant's device. Users can then enter their reactions and additional opinions, which are then sent back to the server. The server then receives this information, and the AI ​​analyzes and generates additional questions and suggestions.

[0715] 5. Integration and Conclusion

[0716] Finally, the server integrates all collected opinions and responses and compiles the final conclusion or results of the discussion, which are presented to the participants' devices and, if necessary, stored in a database.

[0717] Specific examples

[0718] For example, consider a case where a company is conducting remote training and group work is conducted on the theme of "improving marketing strategies."

[0719] 1. Session start: The server launches the generative AI model and completes preparations for the session.

[0720] 2. Collecting participant information: Participants input their opinions, such as "I would like to improve the marketing of our company's product A." This is sent to the server.

[0721] 3. Generative AI opinion generation: Generative AI generates opinions that suggest "effective promotion strategies using social media."

[0722] 4. Presenting opinions and deepening the discussion: The suggestions are displayed on participants' devices, and they can input follow-up questions such as, "Which specific platform would be best?" The AI ​​then generates more detailed suggestions, such as, "Instagram and TikTok are suitable for the target demographic."

[0723] 5. Integration and summary of opinions: The server integrates all opinions, compiles a final proposal for improving the marketing strategy, and presents it to the participants.

[0724] The above is an embodiment of the present invention. This system promotes the exchange of diverse opinions and in-depth discussions even in a remote environment, and realizes effective personnel training.

[0725] The processing flow will be explained below.

[0726] Step 1:

[0727] The user (administrator) inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server.

[0728] Step 2:

[0729] The server receives the session start instruction, launches the generative AI model, completes the initial session setup, and sends a session start notification to the participants' devices.

[0730] Step 3:

[0731] Participants fill in the necessary information in the profile information and initial opinion input form displayed on their device. The participant sends this information from their device, and the server stores the received information in a database.

[0732] Step 4:

[0733] The server analyzes the participant information collected to understand each participant's interests and tendencies. Based on the analysis results, a generative AI model is used to generate a variety of opinions and suggestions.

[0734] Step 5:

[0735] The server sends the generated opinions and suggestions to each participant's device in real time, and the participant can check the presented opinions on their device and enter their reactions or additional opinions.

[0736] Step 6:

[0737] Participants' reactions and additional comments are sent from their devices to the server, which receives this information and instructs the generating AI to analyze it.

[0738] Step 7:

[0739] The AI ​​analyzes participants' responses and generates additional questions and suggestions. The generated questions and suggestions are sent to the server, which then sends them to the participants' devices.

[0740] Step 8:

[0741] Participants input their opinions on the additional questions and suggestions presented on their devices and send them to the server, which receives the information again and analyzes and generates it again as necessary.

[0742] Step 9:

[0743] The server finally aggregates all collected opinions and responses, and summarizes the conclusions and results of the discussion. The results are then displayed on the participants' devices.

[0744] Step 10:

[0745] At the end of the session, the server stores all the results in a database, and the terminal displays a "Session has ended" notification to the participant.

[0746] Example 1

[0747] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0748] To conduct effective human resource training in a remote environment, participants must be able to exchange opinions and hold discussions smoothly. However, in conventional systems, the processes of collecting, generating, and integrating opinions are often inefficient and lack real-time performance. This limits opportunities for participants to share their opinions and engage in in-depth discussions. Therefore, a system that enables effective and smooth exchange of opinions and discussions, even in a remote environment, is needed.

[0749] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0750] In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for collecting participant responses and generating additional questions or suggestions, means for integrating and summarizing the collected opinions and responses, means for activating the generative AI model and initializing the session, means for temporarily storing the generated opinions in the server, and means for presenting the final results to the participants' terminals. This enables efficient and smooth exchange of opinions and discussions even in a remote environment.

[0751] "Participant information" is data including profile information and initial opinions entered by a user.

[0752] A "generative AI model" is an artificial intelligence mechanism that generates diverse opinions and suggestions based on collected participant information.

[0753] "Means for generating opinions" refers to the process and devices that use generative AI models to generate diverse opinions and suggestions.

[0754] The "means for presenting opinions" refers to the process and device for displaying the generated opinions and suggestions on the terminals of the participants.

[0755] The "means for collecting responses" refers to the process and device for collecting responses and additional comments entered by participants in response to generated opinions.

[0756] "Means for generating follow-up questions or suggestions" refers to the process and devices that use a generative AI model to generate new questions or suggestions based on the collected responses.

[0757] A "means for synthesizing opinions and responses" is a process and device for organizing all collected opinions and responses and summarizing the final conclusions and results.

[0758] A "means for initializing a session" is a process and device that performs the necessary settings to begin a training session.

[0759] The "means for temporarily storing opinions on a server" refers to a process and device for storing generated opinions and suggestions on a server for a certain period of time.

[0760] The "means for presenting the final result" refers to the process and device for displaying the integrated final conclusion or the results of the discussion on the terminals of the participants.

[0761] This invention relates to a group work support system using generative AI, and aims to provide effective human resource training in a remote environment. This system includes a server, participant terminals, and a generative AI model, and communication between each element is via a network.

[0762] System Configuration

[0763] This system consists of the following main hardware and software:

[0764] Server: A high-performance computer server that processes data and runs the generative AI model. The main software installed on it is the generative AI model (e.g., GPT-4).

[0765] Participant device: A PC, tablet, or smartphone used by each participant. This device is used to send participant input information to the server and display information sent from the server.

[0766] Network: The network infrastructure for communication between the server and participant devices via the Internet.

[0767] Program processing

[0768] Each process proceeds as follows:

[0769] 1. Start the session

[0770] A command to start a training session is sent from a device managed by the user to the server, which receives the command, launches the generative AI model, and completes the initial setup of the session.

[0771] 2. Collection of Participant Information

[0772] After the initial session setup is complete, the server displays a form for entering profile information and initial opinions on each participant's device. Users enter their own information and initial opinions on their devices and send them to the server. The server stores the received information in a database.

[0773] 3. Generative AI opinion generation

[0774] The server analyzes the collected participant information to understand each participant's interests and tendencies, and uses a generative AI model to generate various opinions and suggestions. The generated opinions are temporarily stored on the server.

[0775] 4. Presenting opinions and deepening discussions

[0776] The server presents the generated opinions and suggestions to each participant's device in real time. Users then input their reactions and additional opinions and send that information back to the server. The server then receives this information and uses the generative AI to generate additional questions and suggestions, which are then presented to the participants.

[0777] 5. Integration and Conclusion

[0778] The server aggregates all collected opinions and responses and compiles the final conclusion or results of the discussion, which are presented to each participant's device and stored in a database if necessary.

[0779] Specific examples

[0780] For example, if a company is conducting remote training and group work on the theme of "improving marketing strategies," it will be used in the following manner.

[0781] 1. Session start: The user sends a session start instruction from the terminal to the server, and the server starts the generation AI and completes the session preparation.

[0782] 2. Collecting participant information: The server displays an information input form on each participant's device, and the user enters their opinion, such as "I would like to improve the marketing of our company's product A," and sends it to the server. The server stores the information in a database.

[0783] 3. Generative AI opinion generation: The server uses a generative AI model based on the collected information to generate opinions such as "effective promotion strategies using social media."

[0784] 4. Presenting opinions and deepening the discussion: The server displays the generated suggestions on each participant's device, and the user inputs additional questions such as "Which specific platform is best?" The AI ​​then generates a suggestion that "Instagram and TikTok are suitable for the target demographic" and presents it again.

[0785] 5. Integration and summary of opinions: The server integrates all opinions, compiles final marketing strategy improvement proposals, and presents them to each participant. The results are stored in a database as needed.

[0786] Prompt Sentence Examples

[0787] "Please use generative AI to provide effective suggestions for improving our marketing strategy. Also, please provide details on the specific platforms you would recommend."

[0788] As described above, this system promotes the exchange of diverse opinions and in-depth discussions even in a remote environment, enabling effective human resource training.

[0789] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0790] Step 1:

[0791] A command to start a training session is sent to the server from a device managed by the user. The input is a "start session" command generated by the user's operation, and the output is the implementation of session initialization on the server side. The server receives this command and launches a generative AI model (e.g., GPT-4). The server performs session initialization, which includes initializing user information and allocating necessary resources. The server logs the completion of the setup and generates a message indicating that it is ready.

[0792] Step 2:

[0793] The server sends a request to each participant's device to display an input form for profile information and initial opinions. The input is the request from the server, and the output is the input form displayed on the device. The device displays the form for the user to input, and the user enters their own information (e.g., name, job title, interests) and initial opinions into the form. When the user clicks the "Submit" button, the input data is sent to the server. The server analyzes the received data and saves it in an SQL database.

[0794] Step 3:

[0795] The server retrieves the collected participant information from the database and analyzes it. The input is the participant information retrieved from the database, and the output is data on the analyzed participants' interests and tendencies. The server provides the analysis results as input to the generative AI model and sends prompts to generate various opinions and suggestions. The generative AI model generates opinions and suggestions and sends them back to the server, which temporarily stores them in cache memory.

[0796] Step 4:

[0797] The server sends the generated opinions and suggestions to each participant's device in real time. The input is opinion data from the generative AI model, and the output is the opinions and suggestions displayed on each participant's device. The device displays this to the user, who then enters their reactions and additional opinions. The information entered by the user is sent to the server. The server analyzes the received reactions and again sends prompts to the generative AI model to generate additional questions or suggestions. The generative AI model generates new suggestions or questions and sends them back to the server. The server again sends this to each participant's device.

[0798] Step 5:

[0799] The server integrates all opinions and responses to compile a final conclusion or discussion result. The input is all collected opinions and responses, and the output is the integrated final result. During the integration process, the server removes duplicate opinions and responses and prioritizes opinions. The resulting final conclusion or proposal is presented to the participants' devices. If necessary, the server stores these results in a database.

[0800] (Application example 1)

[0801] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0802] In order to effectively handle customer service and store operations remotely in a virtual store, a system that can centrally support interactions with diverse customers is required. However, conventional systems have difficulty understanding participants' interests and trends and making appropriate suggestions and responses in real time. They also lack a means to effectively integrate collected opinions and reactions and present optimal responses.

[0803] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0804] In this invention, the server includes a means for collecting participant information, a means for generating opinions using a generative AI model and supporting various interactions in the virtual store, a means for collecting participant responses and generating follow-up questions and suggestions, and a means for presenting the generated opinions to participants in real time and compiling specific countermeasures and suggestions, thereby enabling customer service and store management in the virtual store to be carried out effectively and efficiently.

[0805] "Participant information" refers to data such as the profiles, opinions, and requests of people who use the system, such as customers and staff.

[0806] A "generative AI model" is a model that uses artificial intelligence technology to automatically generate user opinions and suggestions.

[0807] The "opinion generation means" is a function that uses a generative AI model to create opinions and suggestions based on participant information.

[0808] "Presentation means" refers to a method or function for displaying generated opinions and suggestions to the user.

[0809] "Response collection means" is a function for inputting and collecting responses and additional opinions from users.

[0810] "Question generation" is a function that uses a generative AI model to create additional questions and suggestions based on collected responses and data.

[0811] "Integration means" is a function for centralizing and summarizing collected opinions, suggestions, and reactions.

[0812] A "virtual store" is a store that operates on the Internet or in a virtual space and does not have an actual physical location.

[0813] "Interaction support means" is a function that facilitates smooth communication and interaction between customers and staff in a virtual store.

[0814] MODE FOR CARRYING OUT THE INVENTION

[0815] To implement the present invention, the following system configuration and processing method are required.

[0816] System Configuration

[0817] The system of the present invention includes the following elements:

[0818] 1. Hardware

[0819] Server: A central server for running generative AI models and managing the database.

[0820] Device: The device that the user uses as an interface, such as a smartphone, smart glasses, a head-mounted display, or a customer service robot.

[0821] 2. Software

[0822] Generative AI models: Advanced generative AI models such as GPT-4.

[0823] Database System: A database management system such as MySQL.

[0824] Network communication: Communication using HTTP / HTTPS protocols.

[0825] Program processing and data calculation

[0826] 1. Start the session

[0827] The user sends a command to the server to start a session from their device, and the server receives the command, launches the generative AI model, and completes the initial setup.

[0828] 2. Collection of Participant Information

[0829] The server displays a form on each user's device to allow them to enter their profile information and current opinions. The user enters the information and sends it from the device to the server. This information is then stored in a database.

[0830] 3. Opinion generation using generative AI models

[0831] The server analyzes the collected participant information to understand each participant's interests and tendencies. It then uses a generative AI model to generate various opinions and suggestions. The generated opinions are temporarily stored on the server.

[0832] 4. Presenting and discussing opinions

[0833] The generated opinions and suggestions are displayed in real time on the participants' devices. Users can then enter their reactions and additional opinions, which are then sent back to the server. The server then receives this information, and the generative AI model analyzes and generates additional questions and suggestions.

[0834] 5. Integration of opinions and final presentation

[0835] Finally, the server aggregates all collected opinions and responses and compiles final countermeasures and conclusions, which are presented to participants' devices and, if necessary, stored in a database.

[0836] Specific examples

[0837] For example, consider a scenario where a virtual store is promoting a new product, where users provide questions and opinions about the new product, and the generative AI model responds with appropriate suggestions in real time.

[0838] Prompt Sentence Examples

[0839] Start a new product promotion event session. Enter your participant profile information and enter your specific requests and questions. Our AI will then provide you with the best possible suggestions.

[0840] Such a system will enable customer service and store management in virtual stores to be carried out effectively and efficiently.

[0841] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0842] Step 1:

[0843] The user sends an instruction to start a session from their device to the server. The input includes trigger information for starting the session. The server receives this instruction, launches the generative AI model, and completes the initial setup of the session. The launch of the generative AI model is the output.

[0844] Step 2:

[0845] The server displays a form on each user's device for entering profile information and current opinions. The user's profile information and initial opinions are required as input. The user enters the information accordingly and sends it from the device to the server. The server receives this information and stores it in a database. The stored information becomes the output.

[0846] Step 3:

[0847] The server analyzes the collected participant information to understand each participant's interests and tendencies. Participant information stored in a database is required as input. The server uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are output and temporarily stored on the server.

[0848] Step 4:

[0849] The generated opinions and suggestions are presented in real time to the participants' devices. Temporarily saved opinion and suggestion data is required as input. The user then enters their reactions and additional opinions and sends them back to the server. The server receives this information, and the generative AI model analyzes and generates additional questions and suggestions. This is the output.

[0850] Step 5:

[0851] The server integrates all collected opinions and reactions and compiles a final response or conclusion. Resubmitted reactions and additional opinions are required as input. The server integrates the data using a generative AI model to generate a conclusion or response. The generated conclusion is the output, and is displayed on the participants' devices and stored in a database.

[0852] The above is a detailed description of the program's processing steps. In this way, customer service and store operations in the virtual store can be carried out effectively and efficiently.

[0853] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0854] This invention combines a group work support system using generative AI with an emotion engine that recognizes user emotions. The purpose of this system is to provide effective personnel training even in remote environments, and in particular, the emotion engine grasps the emotional state of participants, enhancing the generative AI's ability to generate opinions and progress in discussions.

[0855] System Configuration

[0856] The system includes a server, participant terminals, a generative AI model, and an emotion engine, and communication between these elements is via a network.

[0857] Program processing

[0858] 1. Start the session

[0859] The user (administrator) inputs a command to start a training session into the system. The device receives this input and sends the command to start the session to the server. The server then launches the generative AI model and completes the initial setup of the session.

[0860] 2. Collection of Participant Information

[0861] After the initial setup is complete, the server displays an input form for profile information and initial opinions on each participant's device. Users enter their own information and initial opinions from their device according to this form and send them to the server. The sent information is then stored in a database on the server.

[0862] 3. Collecting Emotional Data

[0863] The device captures participants' emotional data in real time and sends it to a server. For example, it analyzes facial expressions and voice changes using a webcam or microphone. The emotional data is stored on the server and reflected in the generative AI's opinion generation.

[0864] 4. Generative AI opinion generation

[0865] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0866] 5. Presenting opinions and deepening discussions

[0867] The generated opinions and suggestions are displayed in real time on each participant's device. Users then enter their reactions and additional opinions, and this information is sent back to the server. The server receives this information, and the generation AI analyzes and generates additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides the generation AI with appropriate feedback and follow-up questions.

[0868] 6. Integration and Conclusion

[0869] Finally, the server aggregates all collected opinions and responses and summarizes the conclusions and results of the discussion. The results are presented to participants' devices and, if necessary, stored in a database.

[0870] Specific examples

[0871] For example, consider a remote training session in which group work is conducted on the theme of "improving customer service."

[0872] 1. Session start: The server launches the generative AI model and completes the session preparation, including the emotion engine.

[0873] 2. Collecting participant information: Participants input their opinions on how they would like to improve communication with customers. This information is sent to the server.

[0874] 3. Collection of emotional data: Participants' facial expressions and voices are analyzed to collect emotional data such as "current stress level is high."

[0875] 4. Generative AI opinion generation: Based on participants' opinions and emotional data, the generative AI suggests ways to effectively respond to customers while reducing stress.

[0876] 5. Presenting opinions and deepening the discussion: The proposals are displayed on the participants' devices, and they ask, "What are the specific ways to respond?" The emotion engine analyzes the participants' emotional state in response to this question and suggests specific actions to relieve stress.

[0877] 6. Integration and summary of opinions: The server integrates all opinions, compiles a final proposal for improving customer service, and presents it to the participants.

[0878] This completes the implementation of the present invention. This system makes it possible to analyze emotional data using an emotion engine and generate sophisticated opinions using generation AI, enabling effective group work and personnel training even in remote environments.

[0879] The processing flow will be explained below.

[0880] Step 1:

[0881] The user inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server.

[0882] Step 2:

[0883] The server receives the session start instruction, starts the generative AI model and emotion engine, completes the initial session setup, and sends a session start notification to each participant's device.

[0884] Step 3:

[0885] The device displays a form for participants to enter their profile information and initial opinions. Users enter their own information and initial opinions on the device and send them to the server. The sent information is stored in a database.

[0886] Step 4:

[0887] The device collects participants' emotional data in real time. For example, it uses a webcam to recognize facial expressions and a microphone to analyze the tone of voice. The emotional data is then sent to a server.

[0888] Step 5:

[0889] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. Based on the analysis results, the generative AI model generates a variety of opinions and suggestions.

[0890] Step 6:

[0891] The server sends the generated opinions and suggestions to each participant's device in real time, where they are displayed and the user can confirm their content.

[0892] Step 7:

[0893] Users input their reactions to the generated opinions and any additional comments from their devices and send them to the server. The server receives the participants' reaction information, and the generation AI analyzes it again to generate additional questions and suggestions.

[0894] Step 8:

[0895] The device continues to collect the user's emotional data and transmits it to the server, where the emotion engine analyzes it and provides the generative AI with additional suggestions and questions based on the user's emotional state.

[0896] Step 9:

[0897] The server sends new questions and suggestions, including the analysis results of the emotion engine, to participants' devices. Users check these new questions and suggestions on their devices and enter their opinions.

[0898] Step 10:

[0899] The server aggregates all collected opinions and responses and compiles the final conclusion or results of the discussion. This result is sent to the terminal and presented to the user. The server stores the final result in a database.

[0900] The above are the specific processing steps for implementing the present invention. This processing enables emotion analysis using an emotion engine and sophisticated opinion generation using generation AI, enabling effective group work and personnel training even in remote environments.

[0901] Example 2

[0902] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0903] In remote group work and training sessions, communication and exchange of opinions between participants can sometimes be difficult. Therefore, there is a need for a system that can effectively promote interaction in remote environments and grasp and utilize participants' emotions and reactions in real time.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0905] In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for collecting participant responses and generating additional questions or suggestions, means for integrating and summarizing the collected opinions and responses, and means for analyzing participant emotion data and reflecting the results in the generative AI model's opinion generation. This allows opinions and suggestions to be generated and presented taking into account the emotions and interests of participants, even in a remote environment, enabling effective discussion and decision-making.

[0906] The "means for collecting participant information" is a function that collects profile information and initial opinions entered by users through their terminals and transmits them to the server.

[0907] "Means for generating opinions using a generative AI model" refers to a function that uses a generative AI model to create diverse opinions and proposals based on collected participant information and emotional data.

[0908] "Means for presenting generated opinions to participants" refers to a function that displays opinions and suggestions created by the generative AI model on each participant's device in real time.

[0909] The "means of collecting participants' responses and generating additional questions or suggestions" is a function that allows participants to input their responses or additional opinions to the generated opinions and send them to the server to generate new questions or suggestions.

[0910] "Means for integrating and summarizing collected opinions and reactions" is a function that organizes and integrates all opinions and reactions collected on the server and creates a final conclusion or proposal.

[0911] The "means of analyzing emotional data and reflecting it in the opinion generation of the generative AI model" is a function that analyzes participants' emotional data obtained using a webcam or microphone and incorporates the results into the opinion generation process of the generative AI model.

[0912] This invention combines a group work support system using a generative AI model with an emotion engine that recognizes the user's emotions. The purpose of this system is to effectively conduct personnel training even in a remote environment, and in particular, the emotion engine grasps the emotional state of participants, enhancing the generative AI's ability to generate opinions and progress in discussions.

[0913] System Configuration

[0914] The system includes a server, participant devices, a generative AI model, and an emotion engine. Communication between these elements is via a network. Specifically, the user devices are PCs or smartphones, and emotion data is collected using webcams and microphones. The server includes a high-performance computing environment, and the generative AI model uses a natural language generation model such as OpenAI's GPT-3. The emotion engine uses libraries such as OpenCV and TensorFlow to analyze changes in facial expressions and voice.

[0915] Program processing

[0916] The system operates in the following steps:

[0917] 1. Start the session

[0918] The user (administrator) inputs a command to start a training session into the system. The device receives this input and sends the command to start the session to the server. The server then launches the generative AI model and completes the initial setup of the session.

[0919] Example prompt: "Please begin remote training."

[0920] 2. Collection of Participant Information

[0921] Once the initial setup is complete, the server displays a form for entering profile information and initial comments on each participant's device. Users enter their information and initial comments on their device and send them to the server. The sent information is stored in the server's database.

[0922] Example prompt: "Please provide your initial opinion on customer service."

[0923] 3. Collecting Emotional Data

[0924] The device captures participants' emotional data in real time and sends it to a server. For example, it analyzes facial expressions and voice changes using a webcam or microphone. The emotional data is stored on the server and reflected in the generative AI's opinion generation.

[0925] Example prompt: "Collect emotion data by analyzing participants' facial expressions."

[0926] 4. Generative AI opinion generation

[0927] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[0928] Example prompt: "Generate suggestions for how to improve customer service, including ways to reduce stress."

[0929] 5. Presenting opinions and deepening discussions

[0930] The generated opinions and suggestions are displayed in real time on each participant's device. Users then enter their reactions and additional opinions, and this information is sent back to the server. The server receives this information, and the generation AI analyzes and generates additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides the generation AI with appropriate feedback and follow-up questions.

[0931] Example prompt: "Collect participants' reactions to the suggestions presented and generate follow-up questions or suggestions."

[0932] 6. Integration and Conclusion

[0933] Finally, the server aggregates all collected opinions and responses and summarizes the conclusions and results of the discussion. The results are presented to participants' devices and, if necessary, stored in a database.

[0934] Example prompt: "Synthesize all the feedback and come up with a final proposal for improving customer service."

[0935] Specific examples

[0936] For example, consider a remote training session where group work is conducted on the theme of "improving customer service." Specifically, the user clicks the "Start Session" button on the management screen, and the device notifies the server of this operation. The server then activates the generative AI model and emotion engine, completing the initial setup of the session. A participant enters an opinion, such as "I would like to improve communication with customers," and clicks the send button. The device then sends the information to the server, which then stores it in a database.

[0937] Participants' facial expressions and voices are captured using a webcam and microphone and analyzed in real time. The analysis results are sent to a server, which stores them in a database. The server analyzes the participant's information and emotional data and inputs it into a generative AI model (e.g., GPT-3). The generative AI model suggests "ways to effectively respond to customers while reducing stress." The suggestions are displayed on the participant's device, and the participant asks, "What are the specific ways to respond?" The emotion engine analyzes the participant's emotional state in response to the question and suggests specific actions.

[0938] Finally, the server integrates all opinions, compiles final customer service improvement proposals, and presents them to the participants. If necessary, the proposals can be saved in a database. This system enables effective group work and personnel training, even in remote environments.

[0939] The above is an embodiment of the present invention.

[0940] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0941] Step 1:

[0942] The user (administrator) enters an instruction to start a training session into the system. The user clicks the "Start Session" button on a dedicated management screen on the terminal. This action sends an instruction to start the session from the terminal to the server. The terminal sends this information to a specific API endpoint on the server using an HTTP POST request. The server receives this request and executes the startup script for the generative AI model. The server creates an instance of the generative AI model and performs the necessary initial settings.

[0943] Input: User clicks "Start Session" button

[0944] Output: The server initializes an instance of the generated AI model.

[0945] Step 2:

[0946] After the server has finished launching the generated AI model, it displays an input form for profile information and initial opinions on each participant's device. Specifically, the server sends web page data to each device as an HTTP response, which the device parses and displays to the user. The user enters their own information and initial opinions according to the form and clicks the submit button. The device then sends this information to the server via an HTTP POST request in JSON format. The server establishes a database connection and executes an SQL query to store the received information in the database.

[0947] Input: Profile input form sent from the server, information entered by the user

[0948] Output: User information and initial opinion are saved in the server database.

[0949] Step 3:

[0950] The device acquires participants' emotional data in real time. Using input devices such as a webcam and microphone, the device analyzes changes in facial expressions and voice. This analysis is performed using libraries such as OpenCV and TensorFlow. The analysis results are organized in JSON format and sent to the server using an HTTP POST request. The server stores the received emotional data in a database and converts it into the format required for passing it to the generative AI model.

[0951] Input: Video and audio data acquired through the device's webcam and microphone

[0952] Output: Emotion data sent to the server and its converted format

[0953] Step 4:

[0954] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. The server then preprocesses this data to input it into the generative AI. Preprocessing includes data normalization and emotional state scoring. The server then uses the generative AI model to generate a variety of opinions and suggestions. The server inputs the prompt text and preprocessed data into the generative AI model and receives the model's response. The generated opinions are temporarily stored on the server.

[0955] Input: Collected participant information and emotion data

[0956] Output: Opinions and suggestions generated by the generative AI model

[0957] Step 5:

[0958] The server presents the generated opinions and suggestions to each participant's device in real time. The server sends data to each device using WebSocket or HTTP. The user (participant) enters their reaction or additional opinions, and sends this information from their device to the server. The device receives the user's input and sends it back to the server via an HTTP POST request. The server analyzes this and again uses the generation AI to generate additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides appropriate feedback or follow-up questions to the generation AI.

[0959] Input: Opinions generated by a generative AI model or user responses

[0960] Output: Generates additional questions and suggestions, and feedback based on sentiment data

[0961] Step 6:

[0962] The server aggregates all collected opinions and responses. Specifically, it applies an algorithm to compile the best conclusion or discussion result based on the data collected so far. The resulting results are generated as a web page and sent to each device via an HTTP response. The final results and conclusions are stored in a database for later reference and analysis.

[0963] Input: All collected opinions and reactions

[0964] Output: Final conclusions and recommendations are compiled and presented to each device and stored in a database.

[0965] The above are the specific processing steps and operations of this system.

[0966] (Application example 2)

[0967] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0968] In conventional online group work and remote training, it is difficult to properly grasp participants' emotions, which often leads to inefficient discussions and exchanges of opinions. Furthermore, since emotional data is not collected and analyzed in real time, it is difficult to quickly alleviate participants' stress and frustration. In particular, in virtual stores, where interactions with customers take place in real time, it is necessary to immediately grasp customers' emotions and respond appropriately accordingly, but current technology is not sufficient.

[0969] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for acquiring and analyzing emotional data in real time, means for generating responses based on the emotional data using a generative AI model, means for collecting participant reactions and generating follow-up questions or suggestions, and means for integrating and summarizing the collected opinions and reactions. This makes it possible to grasp the emotional states of participants and customers in real time and generate and present appropriate opinions and suggestions based on that data.

[0970] "Participant information" refers to data such as profile information, initial opinions, interests, and tendencies of individuals who participate in training or discussions.

[0971] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate appropriate opinions or suggestions from input data.

[0972] The "means for generating opinions" is a function that uses a generative AI model to generate diverse opinions and proposals based on collected participant information and emotional data.

[0973] The "means for presenting opinions to participants" is a function for displaying generated opinions and suggestions on participants' terminals in real time.

[0974] "Means for acquiring and analyzing emotional data in real time" refers to a function that uses sensor devices such as cameras and microphones to acquire and analyze participants' emotional states from their facial expressions and voices.

[0975] "Means for generating responses based on emotional data" refers to a function that enables the generative AI model to create appropriate responses or suggestions based on the acquired emotional data.

[0976] "Means for collecting responses and generating additional questions or suggestions" is a function for collecting responses and feedback given by participants to the opinions presented and generating further questions or suggestions based on that.

[0977] "Means for integrating and summarizing collected opinions and responses" is a function for integrating the opinions and responses collected from each participant and summarizing the final conclusion or results of the discussion.

[0978] Basic System Configuration

[0979] This invention is a system that includes a server, a user terminal, a generative AI model, and an emotion engine. Communication between these elements is performed via a network.

[0980] Hardware and Software Configuration

[0981] Hardware: smart glasses or head-mounted display (HMD), webcam, microphone, server

[0982] Software: OpenCV, SpeechRecognition, OpenAI API

[0983] Detailed Description

[0984] 1. Server Role

[0985] The server plays a central role in this system, analyzing data collected from each participant's device, running the generative AI model, analyzing emotional data, and managing the generated opinions and suggestions. Specifically, the following data processing is performed:

[0986] Collection and storage of participant information: Participant profile information and initial opinions are sent from the device to the server and stored in a database.

[0987] Emotional data analysis: Emotional data acquired in real time from webcams and microphones is analyzed to understand participants' emotional states.

[0988] Execution of generative AI model: Based on the collected and analyzed data, opinions and questions are generated using the generative AI model.

[0989] 2. Role of the terminal

[0990] The terminal functions as an interface for participants, and various data is input and output.

[0991] Collecting participant information: Participants enter their profile information and initial opinions on their devices.

[0992] Acquiring emotional data: Using the webcam and microphone built into the device, participants' facial expressions and voice data are acquired in real time.

[0993] Displaying opinions and suggestions: The output results of the generative AI model sent from the server are displayed in real time.

[0994] 3. User Operation

[0995] Users input various information into the system and respond to the generated opinions and suggestions. The response data is then sent back to the server for further data analysis and opinion generation.

[0996] Specific examples

[0997] scenario

[0998] For example, consider a case where a customer asks, "Tell me about a new smartphone" in a virtual store. The system's processing flow in this case is as follows:

[0999] 1. Prompt generation

[1000] In this example, the following prompt is created for the generative AI model:

[1001] "The customer seems excited right now. What the customer said: Tell me about your new smartphone.\n\nSuitable response:"

[1002] 2. Acquisition and analysis of emotion data

[1003] The device's webcam and microphone capture the customer's facial expressions and voice, and the emotion engine analyzes them to determine that the customer is "excited."

[1004] 3. AI-based response generation

[1005] Based on the data collected by the server, the generative AI model generates a response such as:

[1006] "You're excited about the features of your new smartphone! Our latest model, the SmartPhone X, is packed with innovative camera technology and a super-fast processor, delivering a next-generation mobile experience."

[1007] In this way, personalized offers can be made in real time within the virtual store, depending on the customer's emotions.

[1008] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1009] Step 1:

[1010] The user inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server. The input is a command to start the session, and the output is to launch the generative AI model and emotion engine and complete the initial setup of the session.

[1011] Step 2:

[1012] The server displays an input form for profile information and initial opinions on each participant's device. Participants enter their own information and initial opinions from their devices and submit them. The input is the participant's profile information and initial opinions, and the output is that this information is sent to the server and stored in a database.

[1013] Step 3:

[1014] The device acquires participants' emotional data in real time and sends it to the server. The emotional data is acquired using a webcam and microphone installed on the device. The input is the participants' facial expressions and voice data, and the output is data on their emotional state obtained by analyzing this data.

[1015] Step 4:

[1016] The server analyzes the collected participant information and emotional data and uses a generative AI model to generate a variety of opinions and suggestions. Specifically, it generates prompts based on the collected data and inputs them into the generative AI model to generate opinions. The input is participant information and emotional data, and the output is the generated opinions and suggestions.

[1017] Step 5:

[1018] The generated opinions and suggestions are presented to each participant's device in real time. Users can then input their reactions and additional opinions. The input is the participant's reaction to the presented opinions, and the output is the response sent back to the server.

[1019] Step 6:

[1020] The server receives the participant's reaction data, and the emotion engine analyzes it to have the generative AI model generate additional questions or suggestions. A prompt sentence is then generated again and input into the generative AI model to generate new suggestions or questions. The input is the participant's reaction data and emotional state, and the output is a newly generated question or suggestion.

[1021] Step 7:

[1022] The server integrates all collected opinions and reactions and compiles the final conclusion or results of the discussion. This is presented to the participants' devices and stored in a database if necessary. The input is the collected opinion and reaction data, and the output is the integrated conclusion or results of the discussion.

[1023] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1024] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1025] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1026] [Fourth embodiment]

[1027] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1028] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1030] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1031] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1032] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1034] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1035] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1036] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1038] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1040] The present invention is a group work support system that uses generative AI. The purpose of this system is to provide effective human resource training even in a remote environment. The following describes in detail the form for implementing this system.

[1041] This system includes a server, participant terminals, and a generative AI model, and communication between these elements is via a network.

[1042] Program processing

[1043] 1. Start the session

[1044] A user-controlled device sends a command to start a training session to the server, which then launches the generative AI model and completes the initial setup of the session.

[1045] 2. Collection of Participant Information

[1046] After the initial setup is complete, the server displays an input form for profile information and initial opinions on each participant's device. Users enter their own information and initial opinions from their device according to this form and send them to the server. The sent information is then stored in a database on the server.

[1047] 3. Generative AI opinion generation

[1048] The server analyzes the collected participant information to understand each participant's interests and tendencies. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[1049] 4. Presenting opinions and deepening discussions

[1050] The generated opinions and suggestions are displayed in real time on each participant's device. Users can then enter their reactions and additional opinions, which are then sent back to the server. The server then receives this information, and the AI ​​analyzes and generates additional questions and suggestions.

[1051] 5. Integration and Conclusion

[1052] Finally, the server integrates all collected opinions and responses and compiles the final conclusion or results of the discussion, which are presented to the participants' devices and, if necessary, stored in a database.

[1053] Specific examples

[1054] For example, consider a case where a company is conducting remote training and group work is conducted on the theme of "improving marketing strategies."

[1055] 1. Session start: The server launches the generative AI model and completes preparations for the session.

[1056] 2. Collecting participant information: Participants input their opinions, such as "I would like to improve the marketing of our company's product A." This is sent to the server.

[1057] 3. Generative AI opinion generation: Generative AI generates opinions that suggest "effective promotion strategies using social media."

[1058] 4. Presenting opinions and deepening the discussion: The suggestions are displayed on participants' devices, and they can input follow-up questions such as, "Which specific platform would be best?" The AI ​​then generates more detailed suggestions, such as, "Instagram and TikTok are suitable for the target demographic."

[1059] 5. Integration and summary of opinions: The server integrates all opinions, compiles a final proposal for improving the marketing strategy, and presents it to the participants.

[1060] The above is an embodiment of the present invention. This system promotes the exchange of diverse opinions and in-depth discussions even in a remote environment, and realizes effective personnel training.

[1061] The processing flow will be explained below.

[1062] Step 1:

[1063] The user (administrator) inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server.

[1064] Step 2:

[1065] The server receives the session start instruction, launches the generative AI model, completes the initial session setup, and sends a session start notification to the participants' devices.

[1066] Step 3:

[1067] Participants fill in the necessary information in the profile information and initial opinion input form displayed on their device. The participant sends this information from their device, and the server stores the received information in a database.

[1068] Step 4:

[1069] The server analyzes the participant information collected to understand each participant's interests and tendencies. Based on the analysis results, a generative AI model is used to generate a variety of opinions and suggestions.

[1070] Step 5:

[1071] The server sends the generated opinions and suggestions to each participant's device in real time, and the participant can check the presented opinions on their device and enter their reactions or additional opinions.

[1072] Step 6:

[1073] Participants' reactions and additional comments are sent from their devices to the server, which receives this information and instructs the generating AI to analyze it.

[1074] Step 7:

[1075] The AI ​​analyzes participants' responses and generates additional questions and suggestions. The generated questions and suggestions are sent to the server, which then sends them to the participants' devices.

[1076] Step 8:

[1077] Participants input their opinions on the additional questions and suggestions presented on their devices and send them to the server, which receives the information again and analyzes and generates it again as necessary.

[1078] Step 9:

[1079] The server finally aggregates all collected opinions and responses, and summarizes the conclusions and results of the discussion. The results are then displayed on the participants' devices.

[1080] Step 10:

[1081] At the end of the session, the server stores all the results in a database, and the terminal displays a "Session has ended" notification to the participant.

[1082] Example 1

[1083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1084] To conduct effective human resource training in a remote environment, participants must be able to exchange opinions and hold discussions smoothly. However, in conventional systems, the processes of collecting, generating, and integrating opinions are often inefficient and lack real-time performance. This limits opportunities for participants to share their opinions and engage in in-depth discussions. Therefore, a system that enables effective and smooth exchange of opinions and discussions, even in a remote environment, is needed.

[1085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1086] In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for collecting participant responses and generating additional questions or suggestions, means for integrating and summarizing the collected opinions and responses, means for activating the generative AI model and initializing the session, means for temporarily storing the generated opinions in the server, and means for presenting the final results to the participants' terminals. This enables efficient and smooth exchange of opinions and discussions even in a remote environment.

[1087] "Participant information" is data including profile information and initial opinions entered by a user.

[1088] A "generative AI model" is an artificial intelligence mechanism that generates diverse opinions and suggestions based on collected participant information.

[1089] "Means for generating opinions" refers to the process and devices that use generative AI models to generate diverse opinions and suggestions.

[1090] The "means for presenting opinions" refers to the process and device for displaying the generated opinions and suggestions on the terminals of the participants.

[1091] The "means for collecting responses" refers to the process and device for collecting responses and additional comments entered by participants in response to generated opinions.

[1092] "Means for generating follow-up questions or suggestions" refers to the process and devices that use a generative AI model to generate new questions or suggestions based on the collected responses.

[1093] A "means for synthesizing opinions and responses" is a process and device for organizing all collected opinions and responses and summarizing the final conclusions and results.

[1094] A "means for initializing a session" is a process and device that performs the necessary settings to begin a training session.

[1095] The "means for temporarily storing opinions on a server" refers to a process and device for storing generated opinions and suggestions on a server for a certain period of time.

[1096] The "means for presenting the final result" refers to the process and device for displaying the integrated final conclusion or the results of the discussion on the terminals of the participants.

[1097] This invention relates to a group work support system using generative AI, and aims to provide effective human resource training in a remote environment. This system includes a server, participant terminals, and a generative AI model, and communication between each element is via a network.

[1098] System Configuration

[1099] This system consists of the following main hardware and software:

[1100] Server: A high-performance computer server that processes data and runs the generative AI model. The main software installed on it is the generative AI model (e.g., GPT-4).

[1101] Participant device: A PC, tablet, or smartphone used by each participant. This device is used to send participant input information to the server and display information sent from the server.

[1102] Network: The network infrastructure for communication between the server and participant devices via the Internet.

[1103] Program processing

[1104] Each process proceeds as follows:

[1105] 1. Start the session

[1106] A command to start a training session is sent from a device managed by the user to the server, which receives the command, launches the generative AI model, and completes the initial setup of the session.

[1107] 2. Collection of Participant Information

[1108] After the initial session setup is complete, the server displays a form for entering profile information and initial opinions on each participant's device. Users enter their own information and initial opinions on their devices and send them to the server. The server stores the received information in a database.

[1109] 3. Generative AI opinion generation

[1110] The server analyzes the collected participant information to understand each participant's interests and tendencies, and uses a generative AI model to generate various opinions and suggestions. The generated opinions are temporarily stored on the server.

[1111] 4. Presenting opinions and deepening discussions

[1112] The server presents the generated opinions and suggestions to each participant's device in real time. Users then input their reactions and additional opinions and send that information back to the server. The server then receives this information and uses the generative AI to generate additional questions and suggestions, which are then presented to the participants.

[1113] 5. Integration and Conclusion

[1114] The server aggregates all collected opinions and responses and compiles the final conclusion or results of the discussion, which are presented to each participant's device and stored in a database if necessary.

[1115] Specific examples

[1116] For example, if a company is conducting remote training and group work on the theme of "improving marketing strategies," it will be used in the following manner.

[1117] 1. Session start: The user sends a session start instruction from the terminal to the server, and the server starts the generation AI and completes the session preparation.

[1118] 2. Collecting participant information: The server displays an information input form on each participant's device, and the user enters their opinion, such as "I would like to improve the marketing of our company's product A," and sends it to the server. The server stores the information in a database.

[1119] 3. Generative AI opinion generation: The server uses a generative AI model based on the collected information to generate opinions such as "effective promotion strategies using social media."

[1120] 4. Presenting opinions and deepening the discussion: The server displays the generated suggestions on each participant's device, and the user inputs additional questions such as "Which specific platform is best?" The AI ​​then generates a suggestion that "Instagram and TikTok are suitable for the target demographic" and presents it again.

[1121] 5. Integration and summary of opinions: The server integrates all opinions, compiles final marketing strategy improvement proposals, and presents them to each participant. The results are stored in a database as needed.

[1122] Prompt Sentence Examples

[1123] "Please use generative AI to provide effective suggestions for improving our marketing strategy. Also, please provide details on the specific platforms you would recommend."

[1124] As described above, this system promotes the exchange of diverse opinions and in-depth discussions even in a remote environment, enabling effective human resource training.

[1125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1126] Step 1:

[1127] A command to start a training session is sent to the server from a device managed by the user. The input is a "start session" command generated by the user's operation, and the output is the implementation of session initialization on the server side. The server receives this command and launches a generative AI model (e.g., GPT-4). The server performs session initialization, which includes initializing user information and allocating necessary resources. The server logs the completion of the setup and generates a message indicating that it is ready.

[1128] Step 2:

[1129] The server sends a request to each participant's device to display an input form for profile information and initial opinions. The input is the request from the server, and the output is the input form displayed on the device. The device displays the form for the user to input, and the user enters their own information (e.g., name, job title, interests) and initial opinions into the form. When the user clicks the "Submit" button, the input data is sent to the server. The server analyzes the received data and saves it in an SQL database.

[1130] Step 3:

[1131] The server retrieves the collected participant information from the database and analyzes it. The input is the participant information retrieved from the database, and the output is data on the analyzed participants' interests and tendencies. The server provides the analysis results as input to the generative AI model and sends prompts to generate various opinions and suggestions. The generative AI model generates opinions and suggestions and sends them back to the server, which temporarily stores them in cache memory.

[1132] Step 4:

[1133] The server sends the generated opinions and suggestions to each participant's device in real time. The input is opinion data from the generative AI model, and the output is the opinions and suggestions displayed on each participant's device. The device displays this to the user, who then enters their reactions and additional opinions. The information entered by the user is sent to the server. The server analyzes the received reactions and again sends prompts to the generative AI model to generate additional questions or suggestions. The generative AI model generates new suggestions or questions and sends them back to the server. The server again sends this to each participant's device.

[1134] Step 5:

[1135] The server integrates all opinions and responses to compile a final conclusion or discussion result. The input is all collected opinions and responses, and the output is the integrated final result. During the integration process, the server removes duplicate opinions and responses and prioritizes opinions. The resulting final conclusion or proposal is presented to the participants' devices. If necessary, the server stores these results in a database.

[1136] (Application example 1)

[1137] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1138] In order to effectively handle customer service and store operations remotely in a virtual store, a system that can centrally support interactions with diverse customers is required. However, conventional systems have difficulty understanding participants' interests and trends and making appropriate suggestions and responses in real time. They also lack a means to effectively integrate collected opinions and reactions and present optimal responses.

[1139] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1140] In this invention, the server includes a means for collecting participant information, a means for generating opinions using a generative AI model and supporting various interactions in the virtual store, a means for collecting participant responses and generating follow-up questions and suggestions, and a means for presenting the generated opinions to participants in real time and compiling specific countermeasures and suggestions, thereby enabling customer service and store management in the virtual store to be carried out effectively and efficiently.

[1141] "Participant information" refers to data such as the profiles, opinions, and requests of people who use the system, such as customers and staff.

[1142] A "generative AI model" is a model that uses artificial intelligence technology to automatically generate user opinions and suggestions.

[1143] The "opinion generation means" is a function that uses a generative AI model to create opinions and suggestions based on participant information.

[1144] "Presentation means" refers to a method or function for displaying generated opinions and suggestions to the user.

[1145] "Response collection means" is a function for inputting and collecting responses and additional opinions from users.

[1146] "Question generation" is a function that uses a generative AI model to create additional questions and suggestions based on collected responses and data.

[1147] "Integration means" is a function for centralizing and summarizing collected opinions, suggestions, and reactions.

[1148] A "virtual store" is a store that operates on the Internet or in a virtual space and does not have an actual physical location.

[1149] "Interaction support means" is a function that facilitates smooth communication and interaction between customers and staff in a virtual store.

[1150] MODE FOR CARRYING OUT THE INVENTION

[1151] To implement the present invention, the following system configuration and processing method are required.

[1152] System Configuration

[1153] The system of the present invention includes the following elements:

[1154] 1. Hardware

[1155] Server: A central server for running generative AI models and managing the database.

[1156] Device: The device that the user uses as an interface, such as a smartphone, smart glasses, a head-mounted display, or a customer service robot.

[1157] 2. Software

[1158] Generative AI models: Advanced generative AI models such as GPT-4.

[1159] Database System: A database management system such as MySQL.

[1160] Network communication: Communication using HTTP / HTTPS protocols.

[1161] Program processing and data calculation

[1162] 1. Start the session

[1163] The user sends a command to the server to start a session from their device, and the server receives the command, launches the generative AI model, and completes the initial setup.

[1164] 2. Collection of Participant Information

[1165] The server displays a form on each user's device to allow them to enter their profile information and current opinions. The user enters the information and sends it from the device to the server. This information is then stored in a database.

[1166] 3. Opinion generation using generative AI models

[1167] The server analyzes the collected participant information to understand each participant's interests and tendencies. It then uses a generative AI model to generate various opinions and suggestions. The generated opinions are temporarily stored on the server.

[1168] 4. Presenting and discussing opinions

[1169] The generated opinions and suggestions are displayed in real time on the participants' devices. Users can then enter their reactions and additional opinions, which are then sent back to the server. The server then receives this information, and the generative AI model analyzes and generates additional questions and suggestions.

[1170] 5. Integration of opinions and final presentation

[1171] Finally, the server aggregates all collected opinions and responses and compiles final countermeasures and conclusions, which are presented to participants' devices and, if necessary, stored in a database.

[1172] Specific examples

[1173] For example, consider a scenario where a virtual store is promoting a new product, where users provide questions and opinions about the new product, and the generative AI model responds with appropriate suggestions in real time.

[1174] Prompt Sentence Examples

[1175] Start a new product promotion event session. Enter your participant profile information and enter your specific requests and questions. Our AI will then provide you with the best possible suggestions.

[1176] Such a system will enable customer service and store management in virtual stores to be carried out effectively and efficiently.

[1177] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1178] Step 1:

[1179] The user sends an instruction to start a session from their device to the server. The input includes trigger information for starting the session. The server receives this instruction, launches the generative AI model, and completes the initial setup of the session. The launch of the generative AI model is the output.

[1180] Step 2:

[1181] The server displays a form on each user's device for entering profile information and current opinions. The user's profile information and initial opinions are required as input. The user enters the information accordingly and sends it from the device to the server. The server receives this information and stores it in a database. The stored information becomes the output.

[1182] Step 3:

[1183] The server analyzes the collected participant information to understand each participant's interests and tendencies. Participant information stored in a database is required as input. The server uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are output and temporarily stored on the server.

[1184] Step 4:

[1185] The generated opinions and suggestions are presented in real time to the participants' devices. Temporarily saved opinion and suggestion data is required as input. The user then enters their reactions and additional opinions and sends them back to the server. The server receives this information, and the generative AI model analyzes and generates additional questions and suggestions. This is the output.

[1186] Step 5:

[1187] The server integrates all collected opinions and reactions and compiles a final response or conclusion. Resubmitted reactions and additional opinions are required as input. The server integrates the data using a generative AI model to generate a conclusion or response. The generated conclusion is the output, and is displayed on the participants' devices and stored in a database.

[1188] The above is a detailed description of the program's processing steps. In this way, customer service and store operations in the virtual store can be carried out effectively and efficiently.

[1189] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1190] This invention combines a group work support system using generative AI with an emotion engine that recognizes user emotions. The purpose of this system is to provide effective personnel training even in remote environments, and in particular, the emotion engine grasps the emotional state of participants, enhancing the generative AI's ability to generate opinions and progress in discussions.

[1191] System Configuration

[1192] The system includes a server, participant terminals, a generative AI model, and an emotion engine, and communication between these elements is via a network.

[1193] Program processing

[1194] 1. Start the session

[1195] The user (administrator) inputs a command to start a training session into the system. The device receives this input and sends the command to start the session to the server. The server then launches the generative AI model and completes the initial setup of the session.

[1196] 2. Collection of Participant Information

[1197] After the initial setup is complete, the server displays an input form for profile information and initial opinions on each participant's device. Users enter their own information and initial opinions from their device according to this form and send them to the server. The sent information is then stored in a database on the server.

[1198] 3. Collecting Emotional Data

[1199] The device captures participants' emotional data in real time and sends it to a server. For example, it analyzes facial expressions and voice changes using a webcam or microphone. The emotional data is stored on the server and reflected in the generative AI's opinion generation.

[1200] 4. Generative AI opinion generation

[1201] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[1202] 5. Presenting opinions and deepening discussions

[1203] The generated opinions and suggestions are displayed in real time on each participant's device. Users then enter their reactions and additional opinions, and this information is sent back to the server. The server receives this information, and the generation AI analyzes and generates additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides the generation AI with appropriate feedback and follow-up questions.

[1204] 6. Integration and Conclusion

[1205] Finally, the server aggregates all collected opinions and responses and summarizes the conclusions and results of the discussion. The results are presented to participants' devices and, if necessary, stored in a database.

[1206] Specific examples

[1207] For example, consider a remote training session in which group work is conducted on the theme of "improving customer service."

[1208] 1. Session start: The server launches the generative AI model and completes the session preparation, including the emotion engine.

[1209] 2. Collecting participant information: Participants input their opinions on how they would like to improve communication with customers. This information is sent to the server.

[1210] 3. Collection of emotional data: Participants' facial expressions and voices are analyzed to collect emotional data such as "current stress level is high."

[1211] 4. Generative AI opinion generation: Based on participants' opinions and emotional data, the generative AI suggests ways to effectively respond to customers while reducing stress.

[1212] 5. Presenting opinions and deepening the discussion: The proposals are displayed on the participants' devices, and they ask, "What are the specific ways to respond?" The emotion engine analyzes the participants' emotional state in response to this question and suggests specific actions to relieve stress.

[1213] 6. Integration and summary of opinions: The server integrates all opinions, compiles a final proposal for improving customer service, and presents it to the participants.

[1214] This completes the implementation of the present invention. This system makes it possible to analyze emotional data using an emotion engine and generate sophisticated opinions using generation AI, enabling effective group work and personnel training even in remote environments.

[1215] The processing flow will be explained below.

[1216] Step 1:

[1217] The user inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server.

[1218] Step 2:

[1219] The server receives the session start instruction, starts the generative AI model and emotion engine, completes the initial session setup, and sends a session start notification to each participant's device.

[1220] Step 3:

[1221] The device displays a form for participants to enter their profile information and initial opinions. Users enter their own information and initial opinions on the device and send them to the server. The sent information is stored in a database.

[1222] Step 4:

[1223] The device collects participants' emotional data in real time. For example, it uses a webcam to recognize facial expressions and a microphone to analyze the tone of voice. The emotional data is then sent to a server.

[1224] Step 5:

[1225] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. Based on the analysis results, the generative AI model generates a variety of opinions and suggestions.

[1226] Step 6:

[1227] The server sends the generated opinions and suggestions to each participant's device in real time, where they are displayed and the user can confirm their content.

[1228] Step 7:

[1229] Users input their reactions to the generated opinions and any additional comments from their devices and send them to the server. The server receives the participants' reaction information, and the generation AI analyzes it again to generate additional questions and suggestions.

[1230] Step 8:

[1231] The device continues to collect the user's emotional data and transmits it to the server, where the emotion engine analyzes it and provides the generative AI with additional suggestions and questions based on the user's emotional state.

[1232] Step 9:

[1233] The server sends new questions and suggestions, including the analysis results of the emotion engine, to participants' devices. Users check these new questions and suggestions on their devices and enter their opinions.

[1234] Step 10:

[1235] The server aggregates all collected opinions and responses and compiles the final conclusion or results of the discussion. This result is sent to the terminal and presented to the user. The server stores the final result in a database.

[1236] The above are the specific processing steps for implementing the present invention. This processing enables emotion analysis using an emotion engine and sophisticated opinion generation using generation AI, enabling effective group work and personnel training even in remote environments.

[1237] Example 2

[1238] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1239] In remote group work and training sessions, communication and exchange of opinions between participants can sometimes be difficult. Therefore, there is a need for a system that can effectively promote interaction in remote environments and grasp and utilize participants' emotions and reactions in real time.

[1240] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1241] In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for collecting participant responses and generating additional questions or suggestions, means for integrating and summarizing the collected opinions and responses, and means for analyzing participant emotion data and reflecting the results in the generative AI model's opinion generation. This allows opinions and suggestions to be generated and presented taking into account the emotions and interests of participants, even in a remote environment, enabling effective discussion and decision-making.

[1242] The "means for collecting participant information" is a function that collects profile information and initial opinions entered by users through their terminals and transmits them to the server.

[1243] "Means for generating opinions using a generative AI model" refers to a function that uses a generative AI model to create diverse opinions and proposals based on collected participant information and emotional data.

[1244] "Means for presenting generated opinions to participants" refers to a function that displays opinions and suggestions created by the generative AI model on each participant's device in real time.

[1245] The "means of collecting participants' responses and generating additional questions or suggestions" is a function that allows participants to input their responses or additional opinions to the generated opinions and send them to the server to generate new questions or suggestions.

[1246] "Means for integrating and summarizing collected opinions and reactions" is a function that organizes and integrates all opinions and reactions collected on the server and creates a final conclusion or proposal.

[1247] The "means of analyzing emotional data and reflecting it in the opinion generation of the generative AI model" is a function that analyzes participants' emotional data obtained using a webcam or microphone and incorporates the results into the opinion generation process of the generative AI model.

[1248] This invention combines a group work support system using a generative AI model with an emotion engine that recognizes the user's emotions. The purpose of this system is to effectively conduct personnel training even in a remote environment, and in particular, the emotion engine grasps the emotional state of participants, enhancing the generative AI's ability to generate opinions and progress in discussions.

[1249] System Configuration

[1250] The system includes a server, participant devices, a generative AI model, and an emotion engine. Communication between these elements is via a network. Specifically, the user devices are PCs or smartphones, and emotion data is collected using webcams and microphones. The server includes a high-performance computing environment, and the generative AI model uses a natural language generation model such as OpenAI's GPT-3. The emotion engine uses libraries such as OpenCV and TensorFlow to analyze changes in facial expressions and voice.

[1251] Program processing

[1252] The system operates in the following steps:

[1253] 1. Start the session

[1254] The user (administrator) inputs a command to start a training session into the system. The device receives this input and sends the command to start the session to the server. The server then launches the generative AI model and completes the initial setup of the session.

[1255] Example prompt: "Please begin remote training."

[1256] 2. Collection of Participant Information

[1257] Once the initial setup is complete, the server displays a form for entering profile information and initial comments on each participant's device. Users enter their information and initial comments on their device and send them to the server. The sent information is stored in the server's database.

[1258] Example prompt: "Please provide your initial opinion on customer service."

[1259] 3. Collecting Emotional Data

[1260] The device captures participants' emotional data in real time and sends it to a server. For example, it analyzes facial expressions and voice changes using a webcam or microphone. The emotional data is stored on the server and reflected in the generative AI's opinion generation.

[1261] Example prompt: "Collect emotion data by analyzing participants' facial expressions."

[1262] 4. Generative AI opinion generation

[1263] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. It then uses a generative AI model to generate a variety of opinions and suggestions. The generated opinions are temporarily stored on the server.

[1264] Example prompt: "Generate suggestions for how to improve customer service, including ways to reduce stress."

[1265] 5. Presenting opinions and deepening discussions

[1266] The generated opinions and suggestions are displayed in real time on each participant's device. Users then enter their reactions and additional opinions, and this information is sent back to the server. The server receives this information, and the generation AI analyzes and generates additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides the generation AI with appropriate feedback and follow-up questions.

[1267] Example prompt: "Collect participants' reactions to the suggestions presented and generate follow-up questions or suggestions."

[1268] 6. Integration and Conclusion

[1269] Finally, the server aggregates all collected opinions and responses and summarizes the conclusions and results of the discussion. The results are presented to participants' devices and, if necessary, stored in a database.

[1270] Example prompt: "Synthesize all the feedback and come up with a final proposal for improving customer service."

[1271] Specific examples

[1272] For example, consider a remote training session where group work is conducted on the theme of "improving customer service." Specifically, the user clicks the "Start Session" button on the management screen, and the device notifies the server of this operation. The server then activates the generative AI model and emotion engine, completing the initial setup of the session. A participant enters an opinion, such as "I would like to improve communication with customers," and clicks the send button. The device then sends the information to the server, which then stores it in a database.

[1273] Participants' facial expressions and voices are captured using a webcam and microphone and analyzed in real time. The analysis results are sent to a server, which stores them in a database. The server analyzes the participant's information and emotional data and inputs it into a generative AI model (e.g., GPT-3). The generative AI model suggests "ways to effectively respond to customers while reducing stress." The suggestions are displayed on the participant's device, and the participant asks, "What are the specific ways to respond?" The emotion engine analyzes the participant's emotional state in response to the question and suggests specific actions.

[1274] Finally, the server integrates all opinions, compiles final customer service improvement proposals, and presents them to the participants. If necessary, the proposals can be saved in a database. This system enables effective group work and personnel training, even in remote environments.

[1275] The above is an embodiment of the present invention.

[1276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1277] Step 1:

[1278] The user (administrator) enters an instruction to start a training session into the system. The user clicks the "Start Session" button on a dedicated management screen on the terminal. This action sends an instruction to start the session from the terminal to the server. The terminal sends this information to a specific API endpoint on the server using an HTTP POST request. The server receives this request and executes the startup script for the generative AI model. The server creates an instance of the generative AI model and performs the necessary initial settings.

[1279] Input: User clicks "Start Session" button

[1280] Output: The server initializes an instance of the generated AI model.

[1281] Step 2:

[1282] After the server has finished launching the generated AI model, it displays an input form for profile information and initial opinions on each participant's device. Specifically, the server sends web page data to each device as an HTTP response, which the device parses and displays to the user. The user enters their own information and initial opinions according to the form and clicks the submit button. The device then sends this information to the server via an HTTP POST request in JSON format. The server establishes a database connection and executes an SQL query to store the received information in the database.

[1283] Input: Profile input form sent from the server, information entered by the user

[1284] Output: User information and initial opinion are saved in the server database.

[1285] Step 3:

[1286] The device acquires participants' emotional data in real time. Using input devices such as a webcam and microphone, the device analyzes changes in facial expressions and voice. This analysis is performed using libraries such as OpenCV and TensorFlow. The analysis results are organized in JSON format and sent to the server using an HTTP POST request. The server stores the received emotional data in a database and converts it into the format required for passing it to the generative AI model.

[1287] Input: Video and audio data acquired through the device's webcam and microphone

[1288] Output: Emotion data sent to the server and its converted format

[1289] Step 4:

[1290] The server analyzes the collected participant information and emotional data to understand each participant's interests, tendencies, and emotional state. The server then preprocesses this data to input it into the generative AI. Preprocessing includes data normalization and emotional state scoring. The server then uses the generative AI model to generate a variety of opinions and suggestions. The server inputs the prompt text and preprocessed data into the generative AI model and receives the model's response. The generated opinions are temporarily stored on the server.

[1291] Input: Collected participant information and emotion data

[1292] Output: Opinions and suggestions generated by the generative AI model

[1293] Step 5:

[1294] The server presents the generated opinions and suggestions to each participant's device in real time. The server sends data to each device using WebSocket or HTTP. The user (participant) enters their reaction or additional opinions, and sends this information from their device to the server. The device receives the user's input and sends it back to the server via an HTTP POST request. The server analyzes this and again uses the generation AI to generate additional questions and suggestions. At this time, the emotion engine analyzes the emotional data and provides appropriate feedback or follow-up questions to the generation AI.

[1295] Input: Opinions generated by a generative AI model or user responses

[1296] Output: Generates additional questions and suggestions, and feedback based on sentiment data

[1297] Step 6:

[1298] The server aggregates all collected opinions and responses. Specifically, it applies an algorithm to compile the best conclusion or discussion result based on the data collected so far. The resulting results are generated as a web page and sent to each device via an HTTP response. The final results and conclusions are stored in a database for later reference and analysis.

[1299] Input: All collected opinions and reactions

[1300] Output: Final conclusions and recommendations are compiled and presented to each device and stored in a database.

[1301] The above are the specific processing steps and operations of this system.

[1302] (Application example 2)

[1303] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1304] In conventional online group work and remote training, it is difficult to properly grasp participants' emotions, which often leads to inefficient discussions and exchanges of opinions. Furthermore, since emotional data is not collected and analyzed in real time, it is difficult to quickly alleviate participants' stress and frustration. In particular, in virtual stores, where interactions with customers take place in real time, it is necessary to immediately grasp customers' emotions and respond appropriately accordingly, but current technology is not sufficient.

[1305] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting participant information, means for generating opinions using a generative AI model, means for presenting the generated opinions to participants, means for acquiring and analyzing emotional data in real time, means for generating responses based on the emotional data using a generative AI model, means for collecting participant reactions and generating follow-up questions or suggestions, and means for integrating and summarizing the collected opinions and reactions. This makes it possible to grasp the emotional states of participants and customers in real time and generate and present appropriate opinions and suggestions based on that data.

[1306] "Participant information" refers to data such as profile information, initial opinions, interests, and tendencies of individuals who participate in training or discussions.

[1307] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate appropriate opinions or suggestions from input data.

[1308] The "means for generating opinions" is a function that uses a generative AI model to generate diverse opinions and proposals based on collected participant information and emotional data.

[1309] The "means for presenting opinions to participants" is a function for displaying generated opinions and suggestions on participants' terminals in real time.

[1310] "Means for acquiring and analyzing emotional data in real time" refers to a function that uses sensor devices such as cameras and microphones to acquire and analyze participants' emotional states from their facial expressions and voices.

[1311] "Means for generating responses based on emotional data" refers to a function that enables the generative AI model to create appropriate responses or suggestions based on the acquired emotional data.

[1312] "Means for collecting responses and generating additional questions or suggestions" is a function for collecting responses and feedback given by participants to the opinions presented and generating further questions or suggestions based on that.

[1313] "Means for integrating and summarizing collected opinions and responses" is a function for integrating the opinions and responses collected from each participant and summarizing the final conclusion or results of the discussion.

[1314] Basic System Configuration

[1315] This invention is a system that includes a server, a user terminal, a generative AI model, and an emotion engine. Communication between these elements is performed via a network.

[1316] Hardware and Software Configuration

[1317] Hardware: smart glasses or head-mounted display (HMD), webcam, microphone, server

[1318] Software: OpenCV, SpeechRecognition, OpenAI API

[1319] Detailed Description

[1320] 1. Server Role

[1321] The server plays a central role in this system, analyzing data collected from each participant's device, running the generative AI model, analyzing emotional data, and managing the generated opinions and suggestions. Specifically, the following data processing is performed:

[1322] Collection and storage of participant information: Participant profile information and initial opinions are sent from the device to the server and stored in a database.

[1323] Emotional data analysis: Emotional data acquired in real time from webcams and microphones is analyzed to understand participants' emotional states.

[1324] Execution of generative AI model: Based on the collected and analyzed data, opinions and questions are generated using the generative AI model.

[1325] 2. Role of the terminal

[1326] The terminal functions as an interface for participants, and various data is input and output.

[1327] Collecting participant information: Participants enter their profile information and initial opinions on their devices.

[1328] Acquiring emotional data: Using the webcam and microphone built into the device, participants' facial expressions and voice data are acquired in real time.

[1329] Displaying opinions and suggestions: The output results of the generative AI model sent from the server are displayed in real time.

[1330] 3. User Operation

[1331] Users input various information into the system and respond to the generated opinions and suggestions. The response data is then sent back to the server for further data analysis and opinion generation.

[1332] Specific examples

[1333] scenario

[1334] For example, consider a case where a customer asks, "Tell me about a new smartphone" in a virtual store. The system's processing flow in this case is as follows:

[1335] 1. Prompt generation

[1336] In this example, the following prompt is created for the generative AI model:

[1337] "The customer seems excited right now. What the customer said: Tell me about your new smartphone.\n\nSuitable response:"

[1338] 2. Acquisition and analysis of emotion data

[1339] The device's webcam and microphone capture the customer's facial expressions and voice, and the emotion engine analyzes them to determine that the customer is "excited."

[1340] 3. AI-based response generation

[1341] Based on the data collected by the server, the generative AI model generates a response such as:

[1342] "You're excited about the features of your new smartphone! Our latest model, the SmartPhone X, is packed with innovative camera technology and a super-fast processor, delivering a next-generation mobile experience."

[1343] In this way, personalized offers can be made in real time within the virtual store, depending on the customer's emotions.

[1344] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1345] Step 1:

[1346] The user inputs a command to start a training session into the system. The terminal receives this input and sends a command to start the session to the server. The input is a command to start the session, and the output is to launch the generative AI model and emotion engine and complete the initial setup of the session.

[1347] Step 2:

[1348] The server displays an input form for profile information and initial opinions on each participant's device. Participants enter their own information and initial opinions from their devices and submit them. The input is the participant's profile information and initial opinions, and the output is that this information is sent to the server and stored in a database.

[1349] Step 3:

[1350] The device acquires participants' emotional data in real time and sends it to the server. The emotional data is acquired using a webcam and microphone installed on the device. The input is the participants' facial expressions and voice data, and the output is data on their emotional state obtained by analyzing this data.

[1351] Step 4:

[1352] The server analyzes the collected participant information and emotional data and uses a generative AI model to generate a variety of opinions and suggestions. Specifically, it generates prompts based on the collected data and inputs them into the generative AI model to generate opinions. The input is participant information and emotional data, and the output is the generated opinions and suggestions.

[1353] Step 5:

[1354] The generated opinions and suggestions are presented to each participant's device in real time. Users can then input their reactions and additional opinions. The input is the participant's reaction to the presented opinions, and the output is the response sent back to the server.

[1355] Step 6:

[1356] The server receives the participant's reaction data, and the emotion engine analyzes it to have the generative AI model generate additional questions or suggestions. A prompt sentence is then generated again and input into the generative AI model to generate new suggestions or questions. The input is the participant's reaction data and emotional state, and the output is a newly generated question or suggestion.

[1357] Step 7:

[1358] The server integrates all collected opinions and reactions and compiles the final conclusion or results of the discussion. This is presented to the participants' devices and stored in a database if necessary. The input is the collected opinion and reaction data, and the output is the integrated conclusion or results of the discussion.

[1359] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1360] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1361] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1362] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1363] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1364] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1365] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1366] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1367] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1368] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1369] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1370] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1371] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1372] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1373] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1374] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1375] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1376] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1377] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1378] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1379] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1380] The following is further disclosed regarding the above embodiment.

[1381] (Claim 1)

[1382] a means of collecting participant information;

[1383] a means for generating opinions using a generative AI model;

[1384] a means for presenting the generated opinions to the participants;

[1385] a means of gathering participant responses and generating follow-up questions and suggestions;

[1386] A means of synthesizing and summarizing the collected opinions and responses;

[1387] A system including:

[1388] (Claim 2)

[1389] 2. The system according to claim 1, further comprising means for analyzing participant information and understanding the interests and tendencies of each participant.

[1390] (Claim 3)

[1391] 10. The system of claim 1, further comprising means for presenting generated opinions and suggestions to participants in real time.

[1392] "Example 1"

[1393] (Claim 1)

[1394] a means of collecting participant information;

[1395] a means for generating opinions using a generative AI model;

[1396] a means for presenting the generated opinions to the participants;

[1397] a means of gathering participant responses and generating follow-up questions and suggestions;

[1398] A means of synthesizing and summarizing the collected opinions and responses;

[1399] A means for launching the generative AI model and initializing the session;

[1400] A means for temporarily storing the generated opinions on a server;

[1401] A means for presenting the final results on the participant's device;

[1402] A system including:

[1403] (Claim 2)

[1404] 2. The system according to claim 1, further comprising means for analyzing participant information and understanding the interests and tendencies of each participant.

[1405] (Claim 3)

[1406] 10. The system of claim 1, further comprising means for presenting generated opinions and suggestions to participants in real time.

[1407] "Application Example 1"

[1408] (Claim 1)

[1409] a means of collecting participant information;

[1410] a means for generating opinions using a generative AI model;

[1411] a means for presenting the generated opinions to the participants;

[1412] a means of gathering participant responses and generating follow-up questions and suggestions;

[1413] A means of synthesizing and summarizing the collected opinions and responses;

[1414] A means of supporting various interactions in virtual stores,

[1415] A system including:

[1416] (Claim 2)

[1417] 2. The system according to claim 1, further comprising means for analyzing participant information and understanding the interests and tendencies of each participant.

[1418] (Claim 3)

[1419] 10. The system according to claim 1, further comprising means for presenting the generated opinions and suggestions to participants in real time and compiling specific countermeasures and suggestions.

[1420] "Example 2: Combining Emotion Engines"

[1421] (Claim 1)

[1422] a means of collecting participant information;

[1423] a means for generating opinions using a generative AI model;

[1424] a means for presenting the generated opinions to the participants;

[1425] a means of gathering participant responses and generating follow-up questions and suggestions;

[1426] A means of synthesizing and summarizing the collected opinions and responses;

[1427] A means to analyze participants' emotional data and reflect it in the opinion generation of the generative AI model,

[1428] A system including:

[1429] (Claim 2)

[1430] 10. The system according to claim 1, further comprising means for analyzing participant information to grasp the interests, tendencies and emotional state of each participant.

[1431] (Claim 3)

[1432] 10. The system of claim 1, further comprising means for presenting the generated opinions and suggestions to the participants in real time and generating feedback and follow-up questions based on the sentiment data.

[1433] "Application example 2 when combining emotion engines"

[1434] (Claim 1)

[1435] a means of collecting participant information;

[1436] a means for generating opinions using a generative AI model;

[1437] a means for presenting the generated opinions to the participants;

[1438] a means of gathering participant responses and generating follow-up questions and suggestions;

[1439] A means of synthesizing and summarizing the collected opinions and responses;

[1440] A means of acquiring and analyzing emotion data in real time;

[1441] means for generating a response based on the emotion data using a generative AI model;

[1442] A system including:

[1443] (Claim 2)

[1444] 10. The system according to claim 1, further comprising means for analyzing the participant information and emotion data to understand the interests and tendencies of each participant.

[1445] (Claim 3)

[1446] 10. The system of claim 1, further comprising means for presenting generated opinions and suggestions to participants in real time and means for presenting responses based on the emotion data in real time. [Explanation of symbols]

[1447] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means of collecting participant information; a means for generating opinions using a generative AI model; a means for presenting the generated opinions to the participants; a means of gathering participant responses and generating follow-up questions and suggestions; A means of synthesizing and summarizing the collected opinions and responses; A system including:

2. 2. The system according to claim 1, further comprising means for analyzing participant information and understanding the interests and tendencies of each participant.

3. The system of claim 1 further comprising means for presenting generated opinions and suggestions to participants in real time.

Images