System

Abstract

A system is provided.SOLUTION: This system is provided with a means for setting an interview subject and a hypothesis, a means for allowing a user to instruct interview contents, and for allowing an interviewer agent to execute a virtual interview based on the instruction, a means for repeatedly and additionally executing the interview, and a means for analyzing the obtained interview contents, and for extracting a business insight or an improvement plan.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] User interviews in traditional marketing processes are costly, time-consuming, and labor-intensive, placing a significant burden on many companies. Analyzing the results of each interview and gaining business insights requires additional effort. Furthermore, repeatedly conducting interviews based on different personas is time-consuming and inefficient. The present invention aims to solve these traditional problems and provide a system that enables more efficient and effective interview research. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means. First, it provides a means for setting interview questions and hypotheses. Next, it has a means for the user to specify the interview content, and an interviewer agent conducts a virtual interview based on that. It also has a means for conducting repeated and additional interviews. Furthermore, it provides a means for analyzing the obtained interview content and extracting business insights and improvement proposals. This realizes a system that generates multiple different personas, conducts interviews with them, records the results in real time, and displays them on the user's dashboard.

[0006] An "interview question" is a specific problem or theme that a user seeks to solve through an interview.

[0007] A "hypothesis" is a prediction or premise that the user wants to verify through an interview.

[0008] A "user" is a person or organization that uses the system to set up and direct interviews.

[0009] An "interviewer agent" is an autonomous software agent that conducts a virtual interview based on the user's instructions.

[0010] A "virtual interview" is a simulated interview conducted by an interviewer agent with a virtual subject.

[0011] A "virtual subject" is a fictitious interview subject generated based on a persona.

[0012] A "persona" is a fictional user character with specific characteristics and behavioral patterns.

[0013] "Analysis" is the process of organizing the interview information obtained and finding useful insights and patterns.

[0014] "Business insights" are specific and practical business suggestions obtained from user interviews.

[0015] "Improvement proposals" are specific improvement measures proposed based on the interview results.

[0016] A "dashboard" is a screen or interface that visually displays interview results and analysis results on the system. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] As an embodiment of the present invention, we provide a virtual interview system that utilizes a large-scale language model (LLM) agent. This system is designed according to the following steps:

[0039] 1. Setting interview questions and hypotheses

[0040] The user logs into the system and inputs the purpose of the interview, the tasks, and the desired hypotheses. The server receives this information and saves it as an interview task. This step clarifies the direction the interview requires and the hypotheses to be verified.

[0041] 2. Instructions and implementation of interview content

[0042] The user specifies specific interview questions and answer formats to the server. For example, the user may set questions to check whether there are any points that are difficult to understand when using the app for the first time. The server builds an interview framework based on these questions and sends it to an interviewer agent. The interviewer agent then conducts the interview with the virtual subject and collects the answers.

[0043] 3. Conducting repeated interviews

[0044] The user sets different personas and repeatedly conducts interviews based on each persona. The server generates a virtual subject for each persona and conducts interviews with them to collect diverse data. For example, the same interview can be conducted with persona A (a young working adult) and persona B (an older housewife) to obtain answers from each perspective.

[0045] 4. Summarizing data and extracting business insights

[0046] After all interviews are completed, the server organizes the collected data, which is then summarized by the analyst agent. The analyst agent analyzes the interview results and extracts specific patterns and business insights. The resulting insights and improvement suggestions are displayed on the user's dashboard, allowing them to make real business improvements.

[0047] Specific examples

[0048] Suppose a user wants to evaluate the usability of a new mobile app. The user logs in to the system, sets the interview objective as "evaluating usability," and sets the hypothesis as "the new UI is intuitive to users." Next, the user sets questions to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and asks for free-form responses. Based on this, the server generates an interviewer agent and conducts an interview with the virtual subject.

[0049] The user then sets two personas: a young working person and an elderly housewife. The server repeatedly interviews these personas and stores their responses in a database. Once the interviews are complete, the server uses an analyst agent to analyze the data and discovers that many users report difficulty understanding the navigation system upon first use. Based on this, the analyst agent suggests adding a tutorial guide as an improvement and displays it on the user's dashboard.

[0050] In this way, the system efficiently conducts virtual interviews and provides specific, actionable business insights, thereby improving the efficiency and effectiveness of marketing operations.

[0051] The processing flow will be explained below.

[0052] Step 1:

[0053] A user logs in to the system. The user enters authentication information and accesses the system. The server verifies the user's authentication information, and if authentication is successful, the user is redirected to the dashboard.

[0054] Step 2:

[0055] The user inputs the purpose of the interview and the hypothesis. The user inputs the purpose of the interview (e.g., "Evaluate the usability of a new mobile app") and the hypothesis (e.g., "The new UI is intuitive to users") from the device. The server receives this information and saves it as an interview assignment.

[0056] Step 3:

[0057] The user sets the interview questions and answer format. The user inputs specific questions (e.g., "Which features were difficult to use?") from the terminal and determines the answer format (e.g., free text, multiple choice). The server receives these questions and answer formats and sends them to the interviewer agent.

[0058] Step 4:

[0059] The interviewer agent conducts an interview with the virtual subject. The server creates an interviewer agent and starts interviewing the virtual subject. The interviewer agent presents questions one by one and collects answers from the virtual subject. The server records these answers in real time.

[0060] Step 5:

[0061] Users set different personas. Users set target personas (e.g., young working people, elderly housewives) from their terminals. The server generates virtual targets based on each persona.

[0062] Step 6:

[0063] The server conducts interviews for each persona. The server generates an interviewer agent for each persona and repeatedly conducts interviews with the virtual subject. The answers from this virtual subject are stored in a database.

[0064] Step 7:

[0065] The server summarizes the interview data. After ensuring all interviews are complete, the server sends the collected data to the analyst agent, who then summarizes it.

[0066] Step 8:

[0067] Analyst agents extract business insights. Analyst agents analyze summary data and extract useful insights and improvement suggestions (e.g., discovering the insight that "navigation is difficult to understand on first use").

[0068] Step 9:

[0069] The server displays the insights and improvement suggestions on the user's dashboard. The extracted insights and improvement suggestions are visually displayed on the dashboard for the user to use, allowing the user to take concrete improvement measures based on the interview results.

[0070] This series of steps allows users to conduct interview research efficiently and effectively and improve their business based on the data obtained.

[0071] Example 1

[0072] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0073] With conventional interview systems, it was difficult to design and conduct interviews using virtual subjects, making it difficult to quickly obtain effective business insights. Furthermore, it was often necessary to conduct repeated interviews with different personas, and it was difficult to efficiently analyze the collected data. This resulted in delays in decision-making for marketing and product improvement.

[0074] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0075] In this invention, the server includes: a means for a user to log in and set interview tasks and hypotheses; a means for inputting the purpose, tasks, and expected hypotheses of the interview; a means for an interviewer agent to conduct a virtual interview based on user instructions for the content of the interview; a means for setting different personas and repeatedly conducting interviews based on each persona; a means for the interviewer agent to conduct interviews with virtual subjects and collect responses; a means for saving the collected interview content as data; a means for an analyst agent to organize and analyze the collected data and extract business insights and improvement proposals; and a means for displaying the extracted insights and improvement proposals on the user's dashboard. This enables efficient interview design and implementation using virtual subjects, enabling quick and effective business insights to be obtained.

[0076] "User" refers to the entity that uses the system to design, conduct, and analyze the results of interviews.

[0077] "Server" refers to the computer system that receives input information from users and manages and executes instructions from interviewer agents and analyst agents.

[0078] "Interview questions" refer to the objectives or hypotheses you want to clarify through the interview.

[0079] A "hypothesis" is a prediction or conjecture that you intend to verify through an interview.

[0080] "Interview content" refers to the questions and answer format set during the interview.

[0081] An "interviewer agent" is an artificial intelligence that conducts interviews with virtual subjects and collects their responses.

[0082] "Virtual subject" refers to a virtual interviewee generated using LLM (large-scale language model).

[0083] A "persona" is a fictional character that represents the characteristics and attributes of the person being interviewed.

[0084] "Analyst Agent" refers to an artificial intelligence that organizes and analyzes collected interview results and extracts business insights and improvement proposals.

[0085] A "dashboard" refers to an interface that displays analysis results such as insights and improvement suggestions to users.

[0086] "Business insights" refers to useful business knowledge and insights gained from the interview results.

[0087] "Improvement proposals" refer to specific measures for improving business operations or products that are proposed based on the insights gained.

[0088] The present invention relates to a virtual interview system utilizing a large-scale language model (LLM) agent. This system includes a means for a user to log in and set interview questions and hypotheses, a means for an interviewer agent to conduct virtual interviews based on the interview content specified by the user, a means for repeatedly conducting interviews based on different personas, a means for collecting and analyzing interview results, and a means for displaying the obtained insights and improvement suggestions on the user's dashboard.

[0089] Hardware or software used

[0090] Server: Responsible for setting interview topics and hypotheses, managing and operating interviewer and analyst agents, saving and analyzing interview results, and displaying them on a dashboard. A cloud-based service can be used as a specific server.

[0091] Terminal: The device from which the user accesses the system, such as a PC, tablet, or smartphone.

[0092] Large-scale language models (LLMs): Used to process interviewer and analyst agents. Examples include OpenAI's GPT series.

[0093] Database: Data storage for storing collected interview responses and later analysis.

[0094] Program processing overview

[0095] User login and information entry

[0096] The user logs in to the system and inputs the purpose of the interview, the task, and the desired hypothesis. The server receives this information and saves it as an interview task. For example, suppose the user's goal is to "evaluate the usability of a new mobile app" and sets the hypothesis that "the new UI is intuitive to users."

[0097] Interview content setting and instructions

[0098] The user specifies specific interview questions and answer formats to the server. For example, the server sets questions to check whether there are any points that are difficult to understand when using the app for the first time, and requests free-form answers. The server then sends these to the interview agent and generates a framework for the interview.

[0099] Generate virtual subjects and conduct interviews

[0100] Users can set different personas, for example, a young working person and an elderly housewife. The server uses LLM to generate virtual subjects based on these personas. The interviewer agent then interviews the virtual subjects and collects their responses.

[0101] Collecting and storing interview results

[0102] The server stores the interview results in a database. For example, a young worker may give feedback that the navigation was difficult to understand the first time they used the app.

[0103] Data Summarization and Business Insight Extraction

[0104] After all the interviews are completed, the server organizes the collected data and sends it to the analyst agent, who analyzes the data and extracts specific patterns and business insights. For example, the analyst agent might learn that many users find the navigation difficult to understand the first time they use the service.

[0105] Displaying insights on dashboards

[0106] The server displays the insights obtained from the analyst agent on the user's dashboard. The user can review the insights on the dashboard and consider business improvement measures as needed. For example, a suggested improvement would be to "add a tutorial guide."

[0107] Examples of prompt statements

[0108] "Conduct virtual interviews about the difficulties users experience when using a mobile app for the first time. The target personas are young professionals and older housewives. For each persona, collect specific feedback on the first-time navigation in an open-ended format."

[0109] In this way, the present invention enables efficient virtual interviews and provides specific and practical business insights, thereby improving the efficiency and effectiveness of marketing operations and product improvements.

[0110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0111] Program processing steps

[0112] Step 1: User login and interview information

[0113] Specific behavior:

[0114] 1. The user accesses the system from a terminal and enters their username and password on the login page.

[0115] 2. The server authenticates the entered username and password and, if authentication is successful, redirects the user to the dashboard.

[0116] 3. On the dashboard, the user clicks the "Create a new interview" button and enters the interview purpose, task, and desired hypotheses.

[0117] Input: Username, Password, Interview Objective, Task, Hypothesis

[0118] Data processing: User authentication, saving interview information

[0119] Output: User login status, input information saved as interview assignment

[0120] Step 2: Interview setup and instructions

[0121] Specific behavior:

[0122] 1. Users enter specific interview questions and answer formats on the dashboard.

[0123] For example: "Please tell us about any difficulties you encountered when using the app for the first time."

[0124] 2. The server saves the entered questions and prepares instructions for the interviewer agent.

[0125] Input: Specific question and answer format

[0126] Data processing: saving questions and preparing instructions

[0127] Output: Information saved as interview questions, prepared instructions for the interviewer agent

[0128] Step 3: Generate a virtual subject and conduct interviews

[0129] Specific behavior:

[0130] 1. Users can create different personas on the dashboard.

[0131] Examples: young working people, older housewives.

[0132] 2. The server uses LLM to generate virtual targets based on the configured personas.

[0133] 3. The interviewer agent asks the generated virtual subject questions set by the user.

[0134] Input: Persona settings, questions

[0135] Data processing: Creating virtual subjects and conducting questions and answers

[0136] Output: Generated virtual audience, collected responses

[0137] Step 4: Collect and store interview results

[0138] Specific behavior:

[0139] 1. The interviewer agent sends the answers obtained from the virtual subject to the server.

[0140] 2. The server stores the interview results in a database.

[0141] Example: Feedback from young working adults: "The navigation is difficult to understand when using it for the first time."

[0142] Input: Responses from hypothetical subjects

[0143] Data processing: Saving answers

[0144] Output: Archived interview results data

[0145] Step 5: Summarize the data and extract business insights

[0146] Specific behavior:

[0147] 1. The server organizes the interview results stored in the database and sends them to the analyst agent.

[0148] 2. Analyst agents analyze the data and extract specific patterns and business insights.

[0149] Example: "Many users find the navigation difficult to understand on first use."

[0150] Input: Saved interview result data

[0151] Data processing: Data organization and analysis

[0152] Output: Extracted business insights and improvement suggestions

[0153] Step 6: Display insights in a dashboard

[0154] Specific behavior:

[0155] 1. The server displays insights and improvement suggestions obtained from the analyst agent on the user's dashboard.

[0156] 2. Users review insights on the dashboard and consider business improvement measures as needed.

[0157] Example: Consider adding a tutorial guide.

[0158] Input: Insights and improvement suggestions from analyst agents

[0159] Data processing: Displaying insights and improvement suggestions

[0160] Output: Insights and recommendations displayed on the user's dashboard

[0161] By clearly showing the specific operations and inputs / outputs at each processing step, it is possible to explain the entire processing flow of the system in detail, allowing users to efficiently conduct virtual interviews and obtain specific, actionable business insights.

[0162] (Application example 1)

[0163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] In modern businesses, efficiently collecting customer feedback is essential for improving products and services. However, traditional methods have made it difficult to accurately collect and analyze real-time customer feedback in physical stores. Furthermore, there has been a lack of appropriate tools to instantly analyze collected feedback and utilize it as business insights. An effective system is needed to resolve these issues and improve the quality of store operations.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0166] In this invention, the server includes means for setting interview topics and hypotheses, means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on that content, means for repeating and additionally conducting interviews, means for analyzing the obtained interview content and extracting business insights and improvement proposals, means for displaying the interview content on smart glasses for collecting feedback from customers in physical stores and recording it as audio, and means for converting the audio data into text and transmitting it to a cloud server. This makes it possible to efficiently collect and analyze real-time customer feedback in physical stores and quickly obtain business insights.

[0167] An "interview question" is the purpose and content of questions set based on the information you want to collect and the hypothesis you want to verify in the virtual interview.

[0168] A "hypothesis" is a premise or condition that is tested by an interview and is confirmed or denied through a series of interviews.

[0169] "Interview content" refers to detailed instructions including specific questions and answer formats that the user will ask in the interview.

[0170] A "user" is someone who uses the interview system to set interview questions and hypotheses, instruct the content of the interview, and analyze the collected data.

[0171] An "interviewer agent" is an agent that conducts an interview based on the interview content set for a virtual subject.

[0172] "Virtual subjects" are hypothetical respondents generated for interviews and based on different personas.

[0173] A "persona" is a fictional character with different attributes and backgrounds that represents a specific target user.

[0174] "Smart glasses" are wearable devices that have the ability to display specific information and record audio data in real time.

[0175] A "cloud server" is a remote server that stores, processes, and analyzes data and is accessed over a network.

[0176] "Voice data" refers to voice response information collected by users or store staff through smart glasses.

[0177] "Convert to text" refers to the process of converting collected voice data into text data.

[0178] "Business insights" are insights and findings that are useful for business improvement and decision-making, based on the analysis of collected data.

[0179] The system includes a means for setting interview topics and hypotheses, a means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on that content, a means for conducting repeated and additional interviews, a means for analyzing the obtained interview content and extracting business insights and improvement proposals, a means for displaying the interview content on smart glasses for collecting feedback from customers in physical stores and recording it as audio, and a means for converting the audio data into text and sending it to a cloud server.

[0180] Specific system implementation includes the following elements:

[0181] 1. Hardware and Software Used:

[0182] Smart glasses: A device used to display the interview content and record audio data.

[0183] Cloud server: A remote server (e.g., AWS) that stores, processes, and analyzes data.

[0184] Speech recognition software: Software that converts voice data into text data (e.g., Google Cloud Speech-to-Text API).

[0185] Large-scale language models: Software used to analyze interview results and extract insights (e.g., OpenAI GPT-4).

[0186] Database: A database (e.g. MySQL) to store the collected data.

[0187] 2. System operation process:

[0188] The server first sets the interview topic and hypotheses. Based on this, the user specifies the interview content, and the interviewer agent conducts the interview with the virtual subjects. The virtual subjects are generated based on personas, and different interviews are repeatedly conducted for each persona.

[0189] In a physical store, the customer wears smart glasses, and the interview content is displayed on the glasses. The customer asks questions to the customer, which are recorded as audio data. This audio data is converted into text in real time and sent to a cloud server.

[0190] The cloud server stores the submitted text data and analyzes it using large-scale language models to extract specific patterns and business insights, which are then displayed on the user's dashboard.

[0191] 3. Example:

[0192] Suppose you want to investigate which aspects of a new store layout interest customers and which aspects dissatisfy them after the new layout is implemented. To do this, set the following hypotheses and questions:

[0193] Hypothesis: "The new layout will make customer flow more smoothly."

[0194] question:

[0195] 1. What did you think of the new layout?

[0196] 2. What area of ​​the store appealed to you the most?

[0197] 3. Were there any aspects that you were dissatisfied with?

[0198] Example prompt sentence:

[0199] Customer Feedback Data:

[0200] 1. What did you think of the new layout?

[0201] "The flow of traffic was good and it was easy to shop." "I got a little lost."

[0202] 2. What area of ​​the store appealed to you the most?

[0203] "Fashion corner" "Food corner"

[0204] 3. Were there any aspects that you were dissatisfied with?

[0205] "The area around the cash register was crowded" "The parking lot was small"

[0206] Based on the customer feedback below, summarize the key themes and insights.

[0207] This system efficiently collects customer feedback in physical stores and instantly analyzes the data, enabling quick and specific business insights.

[0208] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0209] Step 1:

[0210] Users log in to the system and set interview questions and hypotheses. These are sent to the cloud server and saved as interview questions. This process clarifies the purpose and direction of the interview.

[0211] Input: Interview objectives, questions, hypotheses

[0212] Output: Saved interview assignments

[0213] Specific operation: Receives user input data, sends it to the cloud server, and stores it in the database.

[0214] Step 2:

[0215] The user provides specific interview questions and answer formats to the cloud server, which then builds a framework for an interviewer agent based on this information, and the interviewer agent then conducts an interview with the virtual subject.

[0216] Input: Interview questions, answers

[0217] Output: Constructed interview framework, collected response data

[0218] Specific operations: Receives user instructions, generates a set of questions for the interviewer agent, and conducts an interview with the virtual subject.

[0219] Step 3:

[0220] The cloud server generates different personas and conducts repeated interviews with virtual subjects based on each persona, thereby collecting feedback from a variety of perspectives.

[0221] Input: Persona information

[0222] Output: A set of response data from each persona

[0223] What it does: Generate multiple personas and gather relevant data by conducting specialized virtual interviews for each.

[0224] Step 4:

[0225] The user wears the smart glasses, and the interview content is displayed on the glasses. Questions are asked to the customer, and their answers are recorded as audio.

[0226] Input: Interview Question

[0227] Output: Recorded audio data

[0228] Specific operation: The interview content is displayed on the smart glasses and the customer's voice responses are recorded.

[0229] Step 5:

[0230] The voice data is converted into text in real time and sent to a cloud server using voice recognition software.

[0231] Input: Audio data

[0232] Output: Text data

[0233] What it does: It uses speech recognition software to convert voice data into text and sends it to a cloud server.

[0234] Step 6:

[0235] The cloud server stores the transmitted text data in a database and analyzes the data using a generative AI model.

[0236] Input: Text data

[0237] Output: Analysis results, business insights

[0238] What it does: Collected text data is stored in a database and patterns and insights are extracted using generative AI models.

[0239] Step 7:

[0240] The server displays the insights and improvement suggestions obtained in real time on the user's dashboard.

[0241] Input: Analysis results

[0242] Output: Business insight dashboard display

[0243] What it does: Updates user dashboards with analysis results, providing real-time business insights.

[0244] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0245] As an embodiment of the present invention, we provide a virtual interview system that utilizes a large-scale language model (LLM) agent and an emotion engine. This system is designed according to the following steps:

[0246] 1. Setting interview questions and hypotheses

[0247] The user logs into the system and inputs the purpose of the interview, the tasks, and the desired hypotheses. The server receives this information and saves it as an interview task. This step clarifies the direction the interview requires and the hypotheses to be verified.

[0248] 2. Instructions and implementation of interview content

[0249] The user specifies specific interview questions and answer formats to the server. For example, the user may set questions to check whether there are any points that are difficult to understand when using the app for the first time. The server builds an interview framework based on these questions and sends it to an interviewer agent. The interviewer agent then conducts the interview with the virtual subject and collects the answers.

[0250] 3. Conducting repeated interviews

[0251] The user sets different personas and repeatedly conducts interviews based on each persona. The server generates a virtual subject for each persona and conducts interviews with them to collect diverse data. For example, the same interview can be conducted with persona A (a young working adult) and persona B (an older housewife) to obtain answers from each perspective.

[0252] 4. Applying the Emotion Engine

[0253] During the interview, the server utilizes an emotion engine to recognize the virtual subject's emotional data in real time. The emotion engine detects the subject's emotional state from their voice tone, choice of words, and reaction speed. This allows the interviewer agent to dynamically adjust the tone and content of questions according to the virtual subject's emotions.

[0254] 5. Summarizing data and extracting business insights

[0255] After all interviews are completed, the server organizes the collected data, and the analyst agent summarizes it. The analyst agent integrates the interview results with sentiment data to extract specific patterns and business insights. The resulting insights and improvement suggestions are displayed on the user's dashboard, allowing them to make real business improvements.

[0256] Specific examples

[0257] Suppose a user wants to evaluate the usability of a new mobile app. The user logs in to the system, sets the interview objective as "evaluating usability," and sets the hypothesis as "the new UI is intuitive to users." Next, the user sets questions to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and asks for free-form responses. Based on this, the server generates an interviewer agent and conducts an interview with the virtual subject.

[0258] The user then sets up two personas: a young working professional and an elderly housewife. The server repeatedly interviews these personas and stores their responses in a database. During the interview, the emotion engine recognizes the virtual subjects' emotions in real time, allowing the interviewer agent to adjust the questions they ask based on the emotional data, thereby gaining deeper insights.

[0259] Once the interviews are complete, the server uses an analyst agent to analyze the data and discover that many users report difficulty with navigation during first-time use. Based on this feedback, including sentiment data, the analyst agent suggests adding a tutorial guide as an improvement and displays it on the user's dashboard.

[0260] In this way, the system efficiently combines virtual interviews and emotion recognition to provide specific, actionable business insights, thereby improving the efficiency and effectiveness of marketing operations.

[0261] The processing flow will be explained below.

[0262] Step 1:

[0263] A user logs in to the system. The user enters their authentication information and accesses the system. The server verifies the user's authentication information, and if authentication is successful, the user is redirected to the user's dashboard.

[0264] Step 2:

[0265] The user inputs the purpose of the interview and the hypothesis from the terminal. The user inputs the purpose of the interview (e.g., "Evaluate the usability of a new mobile app") and the hypothesis (e.g., "The new UI is intuitive to users"). The server receives this information and saves it as an interview assignment.

[0266] Step 3:

[0267] The user sets the interview questions and answer format from the terminal. The user inputs specific questions (e.g., "Which features were difficult to use?") and answer formats (e.g., free text, multiple choice). The server receives these questions and builds the framework for the interview.

[0268] Step 4:

[0269] The server generates an interviewer agent to conduct an interview with the virtual subject. The interviewer agent asks questions to the virtual subject based on the questions received from the server and collects answers. The server records these answers in real time.

[0270] Step 5:

[0271] The user sets different personas from the terminal. The user sets the target persona (e.g., young working people, elderly housewives). The server generates virtual targets based on each persona.

[0272] Step 6:

[0273] The server conducts interviews for each persona. The server generates an interviewer agent for each persona and repeatedly conducts interviews with the virtual subject. The answers from this virtual subject are stored in a database.

[0274] Step 7:

[0275] The server uses an emotion engine to recognize the virtual subject's emotions in real time. The emotion engine detects the subject's emotional state from their tone of voice, choice of words, and reaction speed. During the interview, the emotional data is fed back to the interviewer agent, who dynamically adjusts the tone and content of the questions.

[0276] Step 8:

[0277] The server summarizes the interview data. After ensuring all interviews are complete, the server sends the collected data to the analyst agent, who then summarizes it.

[0278] Step 9:

[0279] Analyst agents extract business insights. Analyst agents analyze summary data and extract useful insights and improvement suggestions (e.g., discovering that "navigation is difficult to understand on first use"). This process also integrates sentiment data for more accurate analysis.

[0280] Step 10:

[0281] The server displays the insights and improvement suggestions on the user's dashboard. The extracted insights and improvement suggestions are visually displayed on the dashboard for the user to use, allowing the user to take concrete improvement measures based on the interview results and sentiment data.

[0282] This series of steps allows users to conduct interview surveys efficiently and effectively, and improve their business based on the obtained data and emotion recognition information.

[0283] Example 2

[0284] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0285] Conventional interview systems are unable to capture the subject's emotions in real time, resulting in less nuanced and reliable answers. They also struggle to collect data from diverse perspectives based on different personas, leading to insufficient extraction of business insights. Furthermore, it is difficult to efficiently analyze the collected data and provide users with immediate feedback and improvement suggestions based on that data.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0287] In this invention, the server includes: means for a user to set interview topics and hypotheses; means for a user to specify interview content and for an interviewer agent to conduct interviews with virtual subjects based on the content; means for conducting repeated and additional interviews; means for utilizing an emotion engine to recognize emotional data of the virtual subjects in real time during the interview and dynamically adjust the content and tone of questions; means for generating multiple different personas and conducting interviews with these personas to collect data; means for recording interview results and emotional data in real time and displaying them on a dashboard on a user's operating terminal; and means for analyzing the obtained interview content and emotional data and extracting business insights and improvement proposals. This makes it possible to recognize the emotions of virtual subjects in real time, collect data from various personas, and provide efficient feedback and business insights.

[0288] The "interview topic" refers to the purpose and theme of the interview, and the hypothesis you want to verify.

[0289] A "virtual subject" is a simulated interviewee generated by a computer system.

[0290] An "interviewer agent" is a program or system that conducts interviews with virtual subjects, posing questions and collecting responses.

[0291] The "emotion engine" is a technology that detects emotional states from voice tone, selected words, reaction speed, etc., and recognizes emotional data in real time.

[0292] A "persona" is a fictional character with specific attributes and behavioral patterns, and is used as a representative model for marketing and product design.

[0293] "Business insights" are insights and suggestions that are useful for specific business improvements and decision-making, obtained by analyzing collected data.

[0294] A "dashboard" is an interface that visually displays collected data and analysis results, allowing users to easily view and operate them.

[0295] A "generative AI model" refers to an artificial intelligence model generated by a machine learning algorithm using a large dataset.

[0296] A "prompt" is an instruction used to ask a specific question or give input to an AI model.

[0297] "Response format" refers to the format in which responses to interview questions are collected, and includes free-form and multiple-choice formats.

[0298] This invention is a virtual interview system that utilizes a large-scale language model (LLM) agent and an emotion engine, which allows users to obtain emotion data of virtual subjects in real time and efficiently conduct interviews based on various personas.

[0299] System Configuration

[0300] The system consists of three elements: a server, a terminal, and a user, and operates as follows.

[0301] 1. User Roles

[0302] The user logs in to the system using a terminal and sets the purpose, task, and hypothesis of the interview, which instructs the interviewer agent on the interview content and sends the interview task to the server.

[0303] 2. Server Roles

[0304] The server receives the interview objectives and hypotheses sent by the user and stores them in a database as interview tasks. Next, it generates an interviewer agent and prepares to conduct an interview with the virtual subject. To do this, it uses a generative AI model. The server utilizes an emotion engine to recognize the virtual subject's emotional data in real time and dynamically adjust the content and tone of the questions. The obtained interview content and emotional data are analyzed by the server and provided to the user as business insights and improvement suggestions.

[0305] 3. Roles of the Interviewer Agent and Emotion Engine

[0306] The interviewer agent conducts interviews with virtual subjects and collects their responses. It also analyzes the virtual subjects' emotional data in real time through an emotion engine, dynamically adjusting the content and tone of the interview. The emotion engine detects emotions from voice tone, choice of words, and reaction speed.

[0307] Specific steps and operations

[0308] 1. User login and interview preparation

[0309] The user logs in to the system from their device and sets the purpose of the interview as "evaluating usability" and the hypothesis as "the new UI is intuitive to users." Next, questions are set to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and responses are asked in the form of free text.

[0310] 2. Create personas and conduct interviews

[0311] The user sets two personas: a young working person and an elderly housewife. The server generates virtual subjects based on these personas, operates an interviewer agent, and repeatedly conducts the same interview with each of them. During this process, the emotion engine recognizes the virtual subjects' emotional data in real time, and the interviewer agent adjusts the questions accordingly.

[0312] 3. Analyzing data and providing business insights

[0313] After the interview is completed, the server sends the collected data to an analyst agent for analysis. The analyst agent integrates the interview results with sentiment data to extract specific patterns and insights. For example, the analyst agent may discover that many users report difficulty with navigation during first-time use and suggest adding a tutorial guide. These insights and improvement suggestions are displayed on the user's dashboard.

[0314] Examples of concrete examples and prompts

[0315] As a concrete example, the following prompt sentence can be used:

[0316] Example prompt sentence:

[0317] "We would like to conduct interviews to evaluate the usability of a new mobile app. The purpose of the interview is to 'evaluate usability', and the hypothesis is that 'the new UI is intuitive to users.' We will set questions to identify any aspects of how to use the app that are difficult to understand upon first use, and will ask for free-form responses. The target personas will be young working people and older housewives."

[0318] With these specific operating procedures and system configurations, the present invention can realize efficient and accurate virtual interviews and the extraction of various business insights.

[0319] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0320] Step 1:

[0321] The user logs into the system from a terminal. The user's ID and password are required as input. The login information is sent to the server, which then references the database for authentication. If this authentication is successful, the user is taken to a screen where they can set the interview topic and hypothesis. Specifically, the user enters their ID and password into the login screen and clicks the "Login" button.

[0322] Step 2:

[0323] The user inputs the purpose, task, and hypothesis of the interview and sends it to the server. The input requires the purpose, task, and hypothesis of the interview. The server receives this information and stores it in the database as an interview task. Specifically, the user inputs the purpose and hypothesis of the interview into the form and presses the "Submit" button.

[0324] Step 3:

[0325] The user specifies the interview content and answer format. The question content and answer format are required as input. The server generates an interview framework based on this and builds an interviewer agent. Specifically, the user inputs the question content and answer format and presses the "Send" button. The server generates a question template using a generative AI model.

[0326] Step 4:

[0327] The server launches an interviewer agent and conducts an interview with the virtual subject. The inputs required are an interview framework and a profile of the virtual subject. The server instructs the interviewer agent on the content of the interview, and the agent asks questions to the virtual subject. Specifically, the server launches the agent and has it begin asking questions to the virtual subject.

[0328] Step 5:

[0329] The server utilizes an emotion engine to recognize the virtual subject's emotional data in real time. Inputs include the virtual subject's tone of voice, selected words, and reaction speed. The server analyzes this data with the emotion engine to detect their emotional state. Specifically, the server integrates the emotion engine into the interviewer agent and collects emotional data during the interview.

[0330] Step 6:

[0331] The interviewer agent dynamically adjusts the content and tone of questions based on the emotional data of the virtual subject. The emotional data and the current question are required as input. The server modifies the question based on the emotional data and sends instructions to the interviewer agent. Specifically, the server dynamically changes the next question using the results of the emotional data analysis.

[0332] Step 7:

[0333] The user sets up different personas. The persona's attributes (age, occupation, gender, etc.) are required as input. The server generates multiple virtual subjects based on this and conducts repeated interviews with each of them. Specifically, the user enters the persona information and presses the "Submit" button.

[0334] Step 8:

[0335] The server analyzes the collected interview content and sentiment data. The collected interview data and sentiment data are required as input. The server uses analyst agents to analyze the data and generate business insights and improvement proposals. Specifically, the server sends the data to the analyst agents and displays the analysis results on a dashboard.

[0336] Step 9:

[0337] Users can view analysis results on the dashboard and check improvement proposals. Analysis result data is required as input. Business insights and improvement proposals are displayed as output. In concrete terms, users access the dashboard and check the analysis results.

[0338] (Application example 2)

[0339] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] Conventional methods for evaluating advertising campaigns have made it difficult to quickly and efficiently collect target audience emotional responses and specific feedback. Furthermore, evaluating the effectiveness of advertising campaigns individually for different personas requires a significant amount of time and effort. The present invention aims to solve these problems by efficiently evaluating advertising campaigns and quickly obtaining specific insights that take into account the target audience's emotional responses.

[0341] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0342] In this invention, the server includes means for setting interview topics and hypotheses, means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content, means for setting a plurality of different personas and repeatedly conducting interviews based on each persona, means for using an emotion engine to recognize emotional data of the virtual subject in real time during the interview and for the interviewer agent to dynamically adjust the tone and content of questions based on the emotion, and means for analyzing the obtained interview content and extracting business insights and improvement proposals. This makes it possible to efficiently evaluate the effectiveness of an advertising campaign while taking into account the emotional responses of the target audience and quickly obtain specific feedback and business insights.

[0343] "Means for setting interview questions and hypotheses" is a function that allows system users to specifically input the objectives and goals of the advertising campaign they wish to evaluate.

[0344] "A means for the user to specify the interview content and for the interviewer agent to conduct a virtual interview based on that" is a function that conducts an interview with a virtual subject based on the questions specified by the user.

[0345] "A means of setting up multiple different personas and repeatedly conducting interviews based on each persona" is a function that virtually sets up different target groups and conducts interviews with each of them to collect data.

[0346] "Using an emotion engine to recognize the emotional data of virtual subjects in real time during an interview, allowing the interviewer agent to dynamically adjust the tone and content of questions based on their emotions" is a function that grasps the emotional state of the subject during the interview and flexibly changes the interview approach accordingly.

[0347] "Means of analyzing the interview content obtained and extracting business insights and improvement proposals" is a function that analyzes the collected interview data and derives specific insights and improvement proposals regarding the effectiveness of advertising campaigns.

[0348] The present invention relates to a system for evaluating the effectiveness of an advertising campaign, which operates in the following manner.

[0349] First, users log in to the system and set the objectives and hypotheses for their advertising campaign. For example, they can set a hypothesis that "the new advertisement will be very popular with young people." This setting information is then stored on the server.

[0350] Next, the user specifies the specific content of the interview. Questions can be set such as "What was your first impression when you saw this advertisement?" or "What emotions did you feel when you saw this advertisement?" and responses can be requested in the form of free description or multiple choice. Based on this, the server generates an interviewer agent and begins interviewing the virtual subject.

[0351] In this process, multiple personas are created to represent different target demographics (for example, "male high school students" and "female university students"), and interviews are conducted repeatedly. This process makes it possible to collect a wide variety of data.

[0352] During the interview, the server uses an emotion engine to recognize the virtual subject's emotional data in real time. The emotion engine detects the subject's emotional state from their voice tone, choice of words, reaction speed, etc. Based on the recognized emotional data, the interviewer agent can dynamically adjust the tone and content of their questions to extract deeper insights.

[0353] Once the interviews are complete, the server organizes the collected data, which is then summarized by an analysis agent. The analysis agent then integrates the interview results with sentiment data to extract specific patterns and business insights. The resulting insights and recommendations are displayed on the user's dashboard, allowing them to use them to improve their actual advertising campaigns.

[0354] This invention allows for efficient evaluation of advertising campaigns that take into account the emotional responses of the target audience. For example, it is possible to quickly obtain specific feedback such as how high school boys felt when they saw the advertisement, or how college girls felt when they saw the advertisement.

[0355] Example prompt sentence:

[0356] "How do high school boys feel when they see this ad?"

[0357] "What emotions did college girls have when they saw this ad?"

[0358] Thus, the present invention is a system that utilizes a large-scale language model and an emotion engine to enable the effectiveness of advertising campaigns to be obtained from virtual audiences that are close to the actual target demographic, thereby maximizing the effectiveness of advertising and providing important insights for formulating more effective advertising strategies for the target demographic.

[0359] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0360] Step 1:

[0361] Users log in to the system from their devices and set the objectives and hypotheses for their advertising campaign. Hypotheses such as "the new advertisement will be very popular with young people" are entered as input, and the server receives and stores this information. Based on this configuration information, the server processes the data to determine the direction of the interviews.

[0362] Step 2:

[0363] The user inputs specific interview content from the terminal. For example, they input questions such as, "What was your first impression when you saw this advertisement?" or "What emotions did you feel when you saw this advertisement?" The server receives these and performs data calculations to pass instructions to the interviewer agent. As a result, a question framework for the interviewer agent is constructed.

[0364] Step 3:

[0365] Users can set different target demographics (personas) on their devices. For example, they can input personas such as "high school boys" or "university girls." The server receives this data and processes it to generate multiple virtual targets based on the specified personas.

[0366] Step 4:

[0367] The interviewer agent conducts interviews with virtual subjects based on instructions from the server. When questions are presented to the subjects, their responses are collected. The input is the question and the virtual subject's prompt, and the output is the collected response data.

[0368] Step 5:

[0369] While the virtual subject is answering, the server uses an emotion engine to recognize and analyze emotional data from the answers in real time. The input is the collected answer data and data on the virtual subject's reactions, and the output is the detected emotional data. Based on this emotional data, the interviewer agent dynamically adjusts the tone and content of the questions.

[0370] Step 6:

[0371] Once the interview is over, the server organizes the collected interview data and sentiment data, and the data is summarized by the analysis agent. The input is the collected response data and sentiment data, and the output is specific patterns and business insights.

[0372] Step 7:

[0373] The server displays the analysis results on the user's dashboard, allowing the user to gain specific insights into the effectiveness of their advertising campaigns and areas for improvement. The input is the analyzed data and business insights, and the output is the resulting information displayed on the dashboard.

[0374] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0375] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0376] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0377] [Second embodiment]

[0378] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0379] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0380] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0381] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0382] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0383] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0384] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0385] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0386] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0387] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0388] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0389] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0390] As an embodiment of the present invention, we provide a virtual interview system that utilizes a large-scale language model (LLM) agent. This system is designed according to the following steps:

[0391] 1. Setting interview questions and hypotheses

[0392] The user logs into the system and inputs the purpose of the interview, the tasks, and the desired hypotheses. The server receives this information and saves it as an interview task. This step clarifies the direction the interview requires and the hypotheses to be verified.

[0393] 2. Instructions and implementation of interview content

[0394] The user specifies specific interview questions and answer formats to the server. For example, the user may set questions to check whether there are any points that are difficult to understand when using the app for the first time. The server builds an interview framework based on these questions and sends it to an interviewer agent. The interviewer agent then conducts the interview with the virtual subject and collects the answers.

[0395] 3. Conducting repeated interviews

[0396] The user sets different personas and repeatedly conducts interviews based on each persona. The server generates a virtual subject for each persona and conducts interviews with them to collect diverse data. For example, the same interview can be conducted with persona A (a young working adult) and persona B (an older housewife) to obtain answers from each perspective.

[0397] 4. Summarizing data and extracting business insights

[0398] After all interviews are completed, the server organizes the collected data, which is then summarized by the analyst agent. The analyst agent analyzes the interview results and extracts specific patterns and business insights. The resulting insights and improvement suggestions are displayed on the user's dashboard, allowing them to make real business improvements.

[0399] Specific examples

[0400] Suppose a user wants to evaluate the usability of a new mobile app. The user logs in to the system, sets the interview objective as "evaluating usability," and sets the hypothesis as "the new UI is intuitive to users." Next, the user sets questions to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and asks for free-form responses. Based on this, the server generates an interviewer agent and conducts an interview with the virtual subject.

[0401] The user then sets two personas: a young working person and an elderly housewife. The server repeatedly interviews these personas and stores their responses in a database. Once the interviews are complete, the server uses an analyst agent to analyze the data and discovers that many users report difficulty understanding the navigation system upon first use. Based on this, the analyst agent suggests adding a tutorial guide as an improvement and displays it on the user's dashboard.

[0402] In this way, the system efficiently conducts virtual interviews and provides specific, actionable business insights, thereby improving the efficiency and effectiveness of marketing operations.

[0403] The processing flow will be explained below.

[0404] Step 1:

[0405] A user logs in to the system. The user enters authentication information and accesses the system. The server verifies the user's authentication information, and if authentication is successful, the user is redirected to the dashboard.

[0406] Step 2:

[0407] The user inputs the purpose of the interview and the hypothesis. The user inputs the purpose of the interview (e.g., "Evaluate the usability of a new mobile app") and the hypothesis (e.g., "The new UI is intuitive to users") from the device. The server receives this information and saves it as an interview assignment.

[0408] Step 3:

[0409] The user sets the interview questions and answer format. The user inputs specific questions (e.g., "Which features were difficult to use?") from the terminal and determines the answer format (e.g., free text, multiple choice). The server receives these questions and answer formats and sends them to the interviewer agent.

[0410] Step 4:

[0411] The interviewer agent conducts an interview with the virtual subject. The server creates an interviewer agent and starts interviewing the virtual subject. The interviewer agent presents questions one by one and collects answers from the virtual subject. The server records these answers in real time.

[0412] Step 5:

[0413] Users set different personas. Users set target personas (e.g., young working people, elderly housewives) from their terminals. The server generates virtual targets based on each persona.

[0414] Step 6:

[0415] The server conducts interviews for each persona. The server generates an interviewer agent for each persona and repeatedly conducts interviews with the virtual subject. The answers from this virtual subject are stored in a database.

[0416] Step 7:

[0417] The server summarizes the interview data. After ensuring all interviews are complete, the server sends the collected data to the analyst agent, who then summarizes it.

[0418] Step 8:

[0419] Analyst agents extract business insights. Analyst agents analyze summary data and extract useful insights and improvement suggestions (e.g., discovering the insight that "navigation is difficult to understand on first use").

[0420] Step 9:

[0421] The server displays the insights and improvement suggestions on the user's dashboard. The extracted insights and improvement suggestions are visually displayed on the dashboard for the user to use, allowing the user to take concrete improvement measures based on the interview results.

[0422] This series of steps allows users to conduct interview research efficiently and effectively and improve their business based on the data obtained.

[0423] Example 1

[0424] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0425] With conventional interview systems, it was difficult to design and conduct interviews using virtual subjects, making it difficult to quickly obtain effective business insights. Furthermore, it was often necessary to conduct repeated interviews with different personas, and it was difficult to efficiently analyze the collected data. This resulted in delays in decision-making for marketing and product improvement.

[0426] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0427] In this invention, the server includes: a means for a user to log in and set interview tasks and hypotheses; a means for inputting the purpose, tasks, and expected hypotheses of the interview; a means for an interviewer agent to conduct a virtual interview based on user instructions for the content of the interview; a means for setting different personas and repeatedly conducting interviews based on each persona; a means for the interviewer agent to conduct interviews with virtual subjects and collect responses; a means for saving the collected interview content as data; a means for an analyst agent to organize and analyze the collected data and extract business insights and improvement proposals; and a means for displaying the extracted insights and improvement proposals on the user's dashboard. This enables efficient interview design and implementation using virtual subjects, enabling quick and effective business insights to be obtained.

[0428] "User" refers to the entity that uses the system to design, conduct, and analyze the results of interviews.

[0429] "Server" refers to the computer system that receives input information from users and manages and executes instructions from interviewer agents and analyst agents.

[0430] "Interview questions" refer to the objectives or hypotheses you want to clarify through the interview.

[0431] A "hypothesis" is a prediction or conjecture that you intend to verify through an interview.

[0432] "Interview content" refers to the questions and answer format set during the interview.

[0433] An "interviewer agent" is an artificial intelligence that conducts interviews with virtual subjects and collects their responses.

[0434] "Virtual subject" refers to a virtual interviewee generated using LLM (large-scale language model).

[0435] A "persona" is a fictional character that represents the characteristics and attributes of the person being interviewed.

[0436] "Analyst Agent" refers to an artificial intelligence that organizes and analyzes collected interview results and extracts business insights and improvement proposals.

[0437] A "dashboard" refers to an interface that displays analysis results such as insights and improvement suggestions to users.

[0438] "Business insights" refers to useful business knowledge and insights gained from the interview results.

[0439] "Improvement proposals" refer to specific measures for improving business operations or products that are proposed based on the insights gained.

[0440] The present invention relates to a virtual interview system utilizing a large-scale language model (LLM) agent. This system includes a means for a user to log in and set interview questions and hypotheses, a means for an interviewer agent to conduct virtual interviews based on the interview content specified by the user, a means for repeatedly conducting interviews based on different personas, a means for collecting and analyzing interview results, and a means for displaying the obtained insights and improvement suggestions on the user's dashboard.

[0441] Hardware or software used

[0442] Server: Responsible for setting interview topics and hypotheses, managing and operating interviewer and analyst agents, saving and analyzing interview results, and displaying them on a dashboard. A cloud-based service can be used as a specific server.

[0443] Terminal: The device from which the user accesses the system, such as a PC, tablet, or smartphone.

[0444] Large-scale language models (LLMs): Used to process interviewer and analyst agents. Examples include OpenAI's GPT series.

[0445] Database: Data storage for storing collected interview responses and later analysis.

[0446] Program processing overview

[0447] User login and information entry

[0448] The user logs in to the system and inputs the purpose of the interview, the task, and the desired hypothesis. The server receives this information and saves it as an interview task. For example, suppose the user's goal is to "evaluate the usability of a new mobile app" and sets the hypothesis that "the new UI is intuitive to users."

[0449] Interview content setting and instructions

[0450] The user specifies specific interview questions and answer formats to the server. For example, the server sets questions to check whether there are any points that are difficult to understand when using the app for the first time, and requests free-form answers. The server then sends these to the interview agent and generates a framework for the interview.

[0451] Generate virtual subjects and conduct interviews

[0452] Users can set different personas, for example, a young working person and an elderly housewife. The server uses LLM to generate virtual subjects based on these personas. The interviewer agent then interviews the virtual subjects and collects their responses.

[0453] Collecting and storing interview results

[0454] The server stores the interview results in a database. For example, a young worker may give feedback that the navigation was difficult to understand the first time they used the app.

[0455] Data Summarization and Business Insight Extraction

[0456] After all the interviews are completed, the server organizes the collected data and sends it to the analyst agent, who analyzes the data and extracts specific patterns and business insights. For example, the analyst agent might learn that many users find the navigation difficult to understand the first time they use the service.

[0457] Displaying insights on dashboards

[0458] The server displays the insights obtained from the analyst agent on the user's dashboard. The user can review the insights on the dashboard and consider business improvement measures as needed. For example, a suggested improvement would be to "add a tutorial guide."

[0459] Examples of prompt statements

[0460] "Conduct virtual interviews about the difficulties users experience when using a mobile app for the first time. The target personas are young professionals and older housewives. For each persona, collect specific feedback on the first-time navigation in an open-ended format."

[0461] In this way, the present invention enables efficient virtual interviews and provides specific and practical business insights, thereby improving the efficiency and effectiveness of marketing operations and product improvements.

[0462] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0463] Program processing steps

[0464] Step 1: User login and interview information

[0465] Specific behavior:

[0466] 1. The user accesses the system from a terminal and enters their username and password on the login page.

[0467] 2. The server authenticates the entered username and password and, if authentication is successful, redirects the user to the dashboard.

[0468] 3. On the dashboard, the user clicks the "Create a new interview" button and enters the interview purpose, task, and desired hypotheses.

[0469] Input: Username, Password, Interview Objective, Task, Hypothesis

[0470] Data processing: User authentication, saving interview information

[0471] Output: User login status, input information saved as interview assignment

[0472] Step 2: Interview setup and instructions

[0473] Specific behavior:

[0474] 1. Users enter specific interview questions and answer formats on the dashboard.

[0475] For example: "Please tell us about any difficulties you encountered when using the app for the first time."

[0476] 2. The server saves the entered questions and prepares instructions for the interviewer agent.

[0477] Input: Specific question and answer format

[0478] Data processing: saving questions and preparing instructions

[0479] Output: Information saved as interview questions, prepared instructions for the interviewer agent

[0480] Step 3: Generate a virtual subject and conduct interviews

[0481] Specific behavior:

[0482] 1. Users can create different personas on the dashboard.

[0483] Examples: young working people, older housewives.

[0484] 2. The server uses LLM to generate virtual targets based on the configured personas.

[0485] 3. The interviewer agent asks the generated virtual subject questions set by the user.

[0486] Input: Persona settings, questions

[0487] Data processing: Creating virtual subjects and conducting questions and answers

[0488] Output: Generated virtual audience, collected responses

[0489] Step 4: Collect and store interview results

[0490] Specific behavior:

[0491] 1. The interviewer agent sends the answers obtained from the virtual subject to the server.

[0492] 2. The server stores the interview results in a database.

[0493] Example: Feedback from young working adults: "The navigation is difficult to understand when using it for the first time."

[0494] Input: Responses from hypothetical subjects

[0495] Data processing: Saving answers

[0496] Output: Archived interview results data

[0497] Step 5: Summarize the data and extract business insights

[0498] Specific behavior:

[0499] 1. The server organizes the interview results stored in the database and sends them to the analyst agent.

[0500] 2. Analyst agents analyze the data and extract specific patterns and business insights.

[0501] Example: "Many users find the navigation difficult to understand on first use."

[0502] Input: Saved interview result data

[0503] Data processing: Data organization and analysis

[0504] Output: Extracted business insights and improvement suggestions

[0505] Step 6: Display insights in a dashboard

[0506] Specific behavior:

[0507] 1. The server displays insights and improvement suggestions obtained from the analyst agent on the user's dashboard.

[0508] 2. Users review insights on the dashboard and consider business improvement measures as needed.

[0509] Example: Consider adding a tutorial guide.

[0510] Input: Insights and improvement suggestions from analyst agents

[0511] Data processing: Displaying insights and improvement suggestions

[0512] Output: Insights and recommendations displayed on the user's dashboard

[0513] By clearly showing the specific operations and inputs / outputs at each processing step, it is possible to explain the entire processing flow of the system in detail, allowing users to efficiently conduct virtual interviews and obtain specific, actionable business insights.

[0514] (Application example 1)

[0515] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0516] In modern businesses, efficiently collecting customer feedback is essential for improving products and services. However, traditional methods have made it difficult to accurately collect and analyze real-time customer feedback in physical stores. Furthermore, there has been a lack of appropriate tools to instantly analyze collected feedback and utilize it as business insights. An effective system is needed to resolve these issues and improve the quality of store operations.

[0517] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0518] In this invention, the server includes means for setting interview topics and hypotheses, means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on that content, means for repeating and additionally conducting interviews, means for analyzing the obtained interview content and extracting business insights and improvement proposals, means for displaying the interview content on smart glasses for collecting feedback from customers in physical stores and recording it as audio, and means for converting the audio data into text and transmitting it to a cloud server. This makes it possible to efficiently collect and analyze real-time customer feedback in physical stores and quickly obtain business insights.

[0519] An "interview question" is the purpose and content of questions set based on the information you want to collect and the hypothesis you want to verify in the virtual interview.

[0520] A "hypothesis" is a premise or condition that is tested by an interview and is confirmed or denied through a series of interviews.

[0521] "Interview content" refers to detailed instructions including specific questions and answer formats that the user will ask in the interview.

[0522] A "user" is someone who uses the interview system to set interview questions and hypotheses, instruct the content of the interview, and analyze the collected data.

[0523] An "interviewer agent" is an agent that conducts an interview based on the interview content set for a virtual subject.

[0524] "Virtual subjects" are hypothetical respondents generated for interviews and based on different personas.

[0525] A "persona" is a fictional character with different attributes and backgrounds that represents a specific target user.

[0526] "Smart glasses" are wearable devices that have the ability to display specific information and record audio data in real time.

[0527] A "cloud server" is a remote server that stores, processes, and analyzes data and is accessed over a network.

[0528] "Voice data" refers to voice response information collected by users or store staff through smart glasses.

[0529] "Convert to text" refers to the process of converting collected voice data into text data.

[0530] "Business insights" are insights and findings that are useful for business improvement and decision-making, based on the analysis of collected data.

[0531] The system includes a means for setting interview topics and hypotheses, a means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on that content, a means for conducting repeated and additional interviews, a means for analyzing the obtained interview content and extracting business insights and improvement proposals, a means for displaying the interview content on smart glasses for collecting feedback from customers in physical stores and recording it as audio, and a means for converting the audio data into text and sending it to a cloud server.

[0532] Specific system implementation includes the following elements:

[0533] 1. Hardware and Software Used:

[0534] Smart glasses: A device used to display the interview content and record audio data.

[0535] Cloud server: A remote server (e.g., AWS) that stores, processes, and analyzes data.

[0536] Speech recognition software: Software that converts voice data into text data (e.g., Google Cloud Speech-to-Text API).

[0537] Large-scale language models: Software used to analyze interview results and extract insights (e.g., OpenAI GPT-4).

[0538] Database: A database (e.g. MySQL) to store the collected data.

[0539] 2. System operation process:

[0540] The server first sets the interview topic and hypotheses. Based on this, the user specifies the interview content, and the interviewer agent conducts the interview with the virtual subjects. The virtual subjects are generated based on personas, and different interviews are repeatedly conducted for each persona.

[0541] In a physical store, the customer wears smart glasses, and the interview content is displayed on the glasses. The customer asks questions to the customer, which are recorded as audio data. This audio data is converted into text in real time and sent to a cloud server.

[0542] The cloud server stores the submitted text data and analyzes it using large-scale language models to extract specific patterns and business insights, which are then displayed on the user's dashboard.

[0543] 3. Example:

[0544] Suppose you want to investigate which aspects of a new store layout interest customers and which aspects dissatisfy them after the new layout is implemented. To do this, set the following hypotheses and questions:

[0545] Hypothesis: "The new layout will make customer flow more smoothly."

[0546] question:

[0547] 1. What did you think of the new layout?

[0548] 2. What area of ​​the store appealed to you the most?

[0549] 3. Were there any aspects that you were dissatisfied with?

[0550] Example prompt sentence:

[0551] Customer Feedback Data:

[0552] 1. What did you think of the new layout?

[0553] "The flow of traffic was good and it was easy to shop." "I got a little lost."

[0554] 2. What area of ​​the store appealed to you the most?

[0555] "Fashion corner" "Food corner"

[0556] 3. Were there any aspects that you were dissatisfied with?

[0557] "The area around the cash register was crowded" "The parking lot was small"

[0558] Based on the customer feedback below, summarize the key themes and insights.

[0559] This system efficiently collects customer feedback in physical stores and instantly analyzes the data, enabling quick and specific business insights.

[0560] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0561] Step 1:

[0562] Users log in to the system and set interview questions and hypotheses. These are sent to the cloud server and saved as interview questions. This process clarifies the purpose and direction of the interview.

[0563] Input: Interview objectives, questions, hypotheses

[0564] Output: Saved interview assignments

[0565] Specific operation: Receives user input data, sends it to the cloud server, and stores it in the database.

[0566] Step 2:

[0567] The user provides specific interview questions and answer formats to the cloud server, which then builds a framework for an interviewer agent based on this information, and the interviewer agent then conducts an interview with the virtual subject.

[0568] Input: Interview questions, answers

[0569] Output: Constructed interview framework, collected response data

[0570] Specific operations: Receives user instructions, generates a set of questions for the interviewer agent, and conducts an interview with the virtual subject.

[0571] Step 3:

[0572] The cloud server generates different personas and conducts repeated interviews with virtual subjects based on each persona, thereby collecting feedback from a variety of perspectives.

[0573] Input: Persona information

[0574] Output: A set of response data from each persona

[0575] What it does: Generate multiple personas and gather relevant data by conducting specialized virtual interviews for each.

[0576] Step 4:

[0577] The user wears the smart glasses, and the interview content is displayed on the glasses. Questions are asked to the customer, and their answers are recorded as audio.

[0578] Input: Interview Question

[0579] Output: Recorded audio data

[0580] Specific operation: The interview content is displayed on the smart glasses and the customer's voice responses are recorded.

[0581] Step 5:

[0582] The voice data is converted into text in real time and sent to a cloud server using voice recognition software.

[0583] Input: Audio data

[0584] Output: Text data

[0585] What it does: It uses speech recognition software to convert voice data into text and sends it to a cloud server.

[0586] Step 6:

[0587] The cloud server stores the transmitted text data in a database and analyzes the data using a generative AI model.

[0588] Input: Text data

[0589] Output: Analysis results, business insights

[0590] What it does: Collected text data is stored in a database and patterns and insights are extracted using generative AI models.

[0591] Step 7:

[0592] The server displays the insights and improvement suggestions obtained in real time on the user's dashboard.

[0593] Input: Analysis results

[0594] Output: Business insight dashboard display

[0595] What it does: Updates user dashboards with analysis results, providing real-time business insights.

[0596] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0597] As an embodiment of the present invention, we provide a virtual interview system that utilizes a large-scale language model (LLM) agent and an emotion engine. This system is designed according to the following steps:

[0598] 1. Setting interview questions and hypotheses

[0599] The user logs into the system and inputs the purpose of the interview, the tasks, and the desired hypotheses. The server receives this information and saves it as an interview task. This step clarifies the direction the interview requires and the hypotheses to be verified.

[0600] 2. Instructions and implementation of interview content

[0601] The user specifies specific interview questions and answer formats to the server. For example, the user may set questions to check whether there are any points that are difficult to understand when using the app for the first time. The server builds an interview framework based on these questions and sends it to an interviewer agent. The interviewer agent then conducts the interview with the virtual subject and collects the answers.

[0602] 3. Conducting repeated interviews

[0603] The user sets different personas and repeatedly conducts interviews based on each persona. The server generates a virtual subject for each persona and conducts interviews with them to collect diverse data. For example, the same interview can be conducted with persona A (a young working adult) and persona B (an older housewife) to obtain answers from each perspective.

[0604] 4. Applying the Emotion Engine

[0605] During the interview, the server utilizes an emotion engine to recognize the virtual subject's emotional data in real time. The emotion engine detects the subject's emotional state from their voice tone, choice of words, and reaction speed. This allows the interviewer agent to dynamically adjust the tone and content of questions according to the virtual subject's emotions.

[0606] 5. Summarizing data and extracting business insights

[0607] After all interviews are completed, the server organizes the collected data, and the analyst agent summarizes it. The analyst agent integrates the interview results with sentiment data to extract specific patterns and business insights. The resulting insights and improvement suggestions are displayed on the user's dashboard, allowing them to make real business improvements.

[0608] Specific examples

[0609] Suppose a user wants to evaluate the usability of a new mobile app. The user logs in to the system, sets the interview objective as "evaluating usability," and sets the hypothesis as "the new UI is intuitive to users." Next, the user sets questions to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and asks for free-form responses. Based on this, the server generates an interviewer agent and conducts an interview with the virtual subject.

[0610] The user then sets up two personas: a young working professional and an elderly housewife. The server repeatedly interviews these personas and stores their responses in a database. During the interview, the emotion engine recognizes the virtual subjects' emotions in real time, allowing the interviewer agent to adjust the questions they ask based on the emotional data, thereby gaining deeper insights.

[0611] Once the interviews are complete, the server uses an analyst agent to analyze the data and discover that many users report difficulty with navigation during first-time use. Based on this feedback, including sentiment data, the analyst agent suggests adding a tutorial guide as an improvement and displays it on the user's dashboard.

[0612] In this way, the system efficiently combines virtual interviews and emotion recognition to provide specific, actionable business insights, thereby improving the efficiency and effectiveness of marketing operations.

[0613] The processing flow will be explained below.

[0614] Step 1:

[0615] A user logs in to the system. The user enters their authentication information and accesses the system. The server verifies the user's authentication information, and if authentication is successful, the user is redirected to the user's dashboard.

[0616] Step 2:

[0617] The user inputs the purpose of the interview and the hypothesis from the terminal. The user inputs the purpose of the interview (e.g., "Evaluate the usability of a new mobile app") and the hypothesis (e.g., "The new UI is intuitive to users"). The server receives this information and saves it as an interview assignment.

[0618] Step 3:

[0619] The user sets the interview questions and answer format from the terminal. The user inputs specific questions (e.g., "Which features were difficult to use?") and answer formats (e.g., free text, multiple choice). The server receives these questions and builds the framework for the interview.

[0620] Step 4:

[0621] The server generates an interviewer agent to conduct an interview with the virtual subject. The interviewer agent asks questions to the virtual subject based on the questions received from the server and collects answers. The server records these answers in real time.

[0622] Step 5:

[0623] The user sets different personas from the terminal. The user sets the target persona (e.g., young working people, elderly housewives). The server generates virtual targets based on each persona.

[0624] Step 6:

[0625] The server conducts interviews for each persona. The server generates an interviewer agent for each persona and repeatedly conducts interviews with the virtual subject. The answers from this virtual subject are stored in a database.

[0626] Step 7:

[0627] The server uses an emotion engine to recognize the virtual subject's emotions in real time. The emotion engine detects the subject's emotional state from their tone of voice, choice of words, and reaction speed. During the interview, the emotional data is fed back to the interviewer agent, who dynamically adjusts the tone and content of the questions.

[0628] Step 8:

[0629] The server summarizes the interview data. After ensuring all interviews are complete, the server sends the collected data to the analyst agent, who then summarizes it.

[0630] Step 9:

[0631] Analyst agents extract business insights. Analyst agents analyze summary data and extract useful insights and improvement suggestions (e.g., discovering that "navigation is difficult to understand on first use"). This process also integrates sentiment data for more accurate analysis.

[0632] Step 10:

[0633] The server displays the insights and improvement suggestions on the user's dashboard. The extracted insights and improvement suggestions are visually displayed on the dashboard for the user to use, allowing the user to take concrete improvement measures based on the interview results and sentiment data.

[0634] This series of steps allows users to conduct interview surveys efficiently and effectively, and improve their business based on the obtained data and emotion recognition information.

[0635] Example 2

[0636] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0637] Conventional interview systems are unable to capture the subject's emotions in real time, resulting in less nuanced and reliable answers. They also struggle to collect data from diverse perspectives based on different personas, leading to insufficient extraction of business insights. Furthermore, it is difficult to efficiently analyze the collected data and provide users with immediate feedback and improvement suggestions based on that data.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0639] In this invention, the server includes: means for a user to set interview topics and hypotheses; means for a user to specify interview content and for an interviewer agent to conduct interviews with virtual subjects based on the content; means for conducting repeated and additional interviews; means for utilizing an emotion engine to recognize emotional data of the virtual subjects in real time during the interview and dynamically adjust the content and tone of questions; means for generating multiple different personas and conducting interviews with these personas to collect data; means for recording interview results and emotional data in real time and displaying them on a dashboard on a user's operating terminal; and means for analyzing the obtained interview content and emotional data and extracting business insights and improvement proposals. This makes it possible to recognize the emotions of virtual subjects in real time, collect data from various personas, and provide efficient feedback and business insights.

[0640] The "interview topic" refers to the purpose and theme of the interview, and the hypothesis you want to verify.

[0641] A "virtual subject" is a simulated interviewee generated by a computer system.

[0642] An "interviewer agent" is a program or system that conducts interviews with virtual subjects, posing questions and collecting responses.

[0643] The "emotion engine" is a technology that detects emotional states from voice tone, selected words, reaction speed, etc., and recognizes emotional data in real time.

[0644] A "persona" is a fictional character with specific attributes and behavioral patterns, and is used as a representative model for marketing and product design.

[0645] "Business insights" are insights and suggestions that are useful for specific business improvements and decision-making, obtained by analyzing collected data.

[0646] A "dashboard" is an interface that visually displays collected data and analysis results, allowing users to easily view and operate them.

[0647] A "generative AI model" refers to an artificial intelligence model generated by a machine learning algorithm using a large dataset.

[0648] A "prompt" is an instruction used to ask a specific question or give input to an AI model.

[0649] "Response format" refers to the format in which responses to interview questions are collected, and includes free-form and multiple-choice formats.

[0650] This invention is a virtual interview system that utilizes a large-scale language model (LLM) agent and an emotion engine, which allows users to obtain emotion data of virtual subjects in real time and efficiently conduct interviews based on various personas.

[0651] System Configuration

[0652] The system consists of three elements: a server, a terminal, and a user, and operates as follows.

[0653] 1. User Roles

[0654] The user logs in to the system using a terminal and sets the purpose, task, and hypothesis of the interview, which instructs the interviewer agent on the interview content and sends the interview task to the server.

[0655] 2. Server Roles

[0656] The server receives the interview objectives and hypotheses sent by the user and stores them in a database as interview tasks. Next, it generates an interviewer agent and prepares to conduct an interview with the virtual subject. To do this, it uses a generative AI model. The server utilizes an emotion engine to recognize the virtual subject's emotional data in real time and dynamically adjust the content and tone of the questions. The obtained interview content and emotional data are analyzed by the server and provided to the user as business insights and improvement suggestions.

[0657] 3. Roles of the Interviewer Agent and Emotion Engine

[0658] The interviewer agent conducts interviews with virtual subjects and collects their responses. It also analyzes the virtual subjects' emotional data in real time through an emotion engine, dynamically adjusting the content and tone of the interview. The emotion engine detects emotions from voice tone, choice of words, and reaction speed.

[0659] Specific steps and operations

[0660] 1. User login and interview preparation

[0661] The user logs in to the system from their device and sets the purpose of the interview as "evaluating usability" and the hypothesis as "the new UI is intuitive to users." Next, questions are set to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and responses are asked in the form of free text.

[0662] 2. Create personas and conduct interviews

[0663] The user sets two personas: a young working person and an elderly housewife. The server generates virtual subjects based on these personas, operates an interviewer agent, and repeatedly conducts the same interview with each of them. During this process, the emotion engine recognizes the virtual subjects' emotional data in real time, and the interviewer agent adjusts the questions accordingly.

[0664] 3. Analyzing data and providing business insights

[0665] After the interview is completed, the server sends the collected data to an analyst agent for analysis. The analyst agent integrates the interview results with sentiment data to extract specific patterns and insights. For example, the analyst agent may discover that many users report difficulty with navigation during first-time use and suggest adding a tutorial guide. These insights and improvement suggestions are displayed on the user's dashboard.

[0666] Examples of concrete examples and prompts

[0667] As a concrete example, the following prompt sentence can be used:

[0668] Example prompt sentence:

[0669] "We would like to conduct interviews to evaluate the usability of a new mobile app. The purpose of the interview is to 'evaluate usability', and the hypothesis is that 'the new UI is intuitive to users.' We will set questions to identify any aspects of how to use the app that are difficult to understand upon first use, and will ask for free-form responses. The target personas will be young working people and older housewives."

[0670] With these specific operating procedures and system configurations, the present invention can realize efficient and accurate virtual interviews and the extraction of various business insights.

[0671] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0672] Step 1:

[0673] The user logs into the system from a terminal. The user's ID and password are required as input. The login information is sent to the server, which then references the database for authentication. If this authentication is successful, the user is taken to a screen where they can set the interview topic and hypothesis. Specifically, the user enters their ID and password into the login screen and clicks the "Login" button.

[0674] Step 2:

[0675] The user inputs the purpose, task, and hypothesis of the interview and sends it to the server. The input requires the purpose, task, and hypothesis of the interview. The server receives this information and stores it in the database as an interview task. Specifically, the user inputs the purpose and hypothesis of the interview into the form and presses the "Submit" button.

[0676] Step 3:

[0677] The user specifies the interview content and answer format. The question content and answer format are required as input. The server generates an interview framework based on this and builds an interviewer agent. Specifically, the user inputs the question content and answer format and presses the "Send" button. The server generates a question template using a generative AI model.

[0678] Step 4:

[0679] The server launches an interviewer agent and conducts an interview with the virtual subject. The inputs required are an interview framework and a profile of the virtual subject. The server instructs the interviewer agent on the content of the interview, and the agent asks questions to the virtual subject. Specifically, the server launches the agent and has it begin asking questions to the virtual subject.

[0680] Step 5:

[0681] The server utilizes an emotion engine to recognize the virtual subject's emotional data in real time. Inputs include the virtual subject's tone of voice, selected words, and reaction speed. The server analyzes this data with the emotion engine to detect their emotional state. Specifically, the server integrates the emotion engine into the interviewer agent and collects emotional data during the interview.

[0682] Step 6:

[0683] The interviewer agent dynamically adjusts the content and tone of questions based on the emotional data of the virtual subject. The emotional data and the current question are required as input. The server modifies the question based on the emotional data and sends instructions to the interviewer agent. Specifically, the server dynamically changes the next question using the results of the emotional data analysis.

[0684] Step 7:

[0685] The user sets up different personas. The persona's attributes (age, occupation, gender, etc.) are required as input. The server generates multiple virtual subjects based on this and conducts repeated interviews with each of them. Specifically, the user enters the persona information and presses the "Submit" button.

[0686] Step 8:

[0687] The server analyzes the collected interview content and sentiment data. The collected interview data and sentiment data are required as input. The server uses analyst agents to analyze the data and generate business insights and improvement proposals. Specifically, the server sends the data to the analyst agents and displays the analysis results on a dashboard.

[0688] Step 9:

[0689] Users can view analysis results on the dashboard and check improvement proposals. Analysis result data is required as input. Business insights and improvement proposals are displayed as output. In concrete terms, users access the dashboard and check the analysis results.

[0690] (Application example 2)

[0691] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0692] Conventional methods for evaluating advertising campaigns have made it difficult to quickly and efficiently collect target audience emotional responses and specific feedback. Furthermore, evaluating the effectiveness of advertising campaigns individually for different personas requires a significant amount of time and effort. The present invention aims to solve these problems by efficiently evaluating advertising campaigns and quickly obtaining specific insights that take into account the target audience's emotional responses.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0694] In this invention, the server includes means for setting interview topics and hypotheses, means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content, means for setting a plurality of different personas and repeatedly conducting interviews based on each persona, means for using an emotion engine to recognize emotional data of the virtual subject in real time during the interview and for the interviewer agent to dynamically adjust the tone and content of questions based on the emotion, and means for analyzing the obtained interview content and extracting business insights and improvement proposals. This makes it possible to efficiently evaluate the effectiveness of an advertising campaign while taking into account the emotional responses of the target audience and quickly obtain specific feedback and business insights.

[0695] "Means for setting interview questions and hypotheses" is a function that allows system users to specifically input the objectives and goals of the advertising campaign they wish to evaluate.

[0696] "A means for the user to specify the interview content and for the interviewer agent to conduct a virtual interview based on that" is a function that conducts an interview with a virtual subject based on the questions specified by the user.

[0697] "A means of setting up multiple different personas and repeatedly conducting interviews based on each persona" is a function that virtually sets up different target groups and conducts interviews with each of them to collect data.

[0698] "Using an emotion engine to recognize the emotional data of virtual subjects in real time during an interview, allowing the interviewer agent to dynamically adjust the tone and content of questions based on their emotions" is a function that grasps the emotional state of the subject during the interview and flexibly changes the interview approach accordingly.

[0699] "Means of analyzing the interview content obtained and extracting business insights and improvement proposals" is a function that analyzes the collected interview data and derives specific insights and improvement proposals regarding the effectiveness of advertising campaigns.

[0700] The present invention relates to a system for evaluating the effectiveness of an advertising campaign, which operates in the following manner.

[0701] First, users log in to the system and set the objectives and hypotheses for their advertising campaign. For example, they can set a hypothesis that "the new advertisement will be very popular with young people." This setting information is then stored on the server.

[0702] Next, the user specifies the specific content of the interview. Questions can be set such as "What was your first impression when you saw this advertisement?" or "What emotions did you feel when you saw this advertisement?" and responses can be requested in the form of free description or multiple choice. Based on this, the server generates an interviewer agent and begins interviewing the virtual subject.

[0703] In this process, multiple personas are created to represent different target demographics (for example, "male high school students" and "female university students"), and interviews are conducted repeatedly. This process makes it possible to collect a wide variety of data.

[0704] During the interview, the server uses an emotion engine to recognize the virtual subject's emotional data in real time. The emotion engine detects the subject's emotional state from their voice tone, choice of words, reaction speed, etc. Based on the recognized emotional data, the interviewer agent can dynamically adjust the tone and content of their questions to extract deeper insights.

[0705] Once the interviews are complete, the server organizes the collected data, which is then summarized by an analysis agent. The analysis agent then integrates the interview results with sentiment data to extract specific patterns and business insights. The resulting insights and recommendations are displayed on the user's dashboard, allowing them to use them to improve their actual advertising campaigns.

[0706] This invention allows for efficient evaluation of advertising campaigns that take into account the emotional responses of the target audience. For example, it is possible to quickly obtain specific feedback such as how high school boys felt when they saw the advertisement, or how college girls felt when they saw the advertisement.

[0707] Example prompt sentence:

[0708] "How do high school boys feel when they see this ad?"

[0709] "What emotions did college girls have when they saw this ad?"

[0710] Thus, the present invention is a system that utilizes a large-scale language model and an emotion engine to enable the effectiveness of advertising campaigns to be obtained from virtual audiences that are close to the actual target demographic, thereby maximizing the effectiveness of advertising and providing important insights for formulating more effective advertising strategies for the target demographic.

[0711] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0712] Step 1:

[0713] Users log in to the system from their devices and set the objectives and hypotheses for their advertising campaign. Hypotheses such as "the new advertisement will be very popular with young people" are entered as input, and the server receives and stores this information. Based on this configuration information, the server processes the data to determine the direction of the interviews.

[0714] Step 2:

[0715] The user inputs specific interview content from the terminal. For example, they input questions such as, "What was your first impression when you saw this advertisement?" or "What emotions did you feel when you saw this advertisement?" The server receives these and performs data calculations to pass instructions to the interviewer agent. As a result, a question framework for the interviewer agent is constructed.

[0716] Step 3:

[0717] Users can set different target demographics (personas) on their devices. For example, they can input personas such as "high school boys" or "university girls." The server receives this data and processes it to generate multiple virtual targets based on the specified personas.

[0718] Step 4:

[0719] The interviewer agent conducts interviews with virtual subjects based on instructions from the server. When questions are presented to the subjects, their responses are collected. The input is the question and the virtual subject's prompt, and the output is the collected response data.

[0720] Step 5:

[0721] While the virtual subject is answering, the server uses an emotion engine to recognize and analyze emotional data from the answers in real time. The input is the collected answer data and data on the virtual subject's reactions, and the output is the detected emotional data. Based on this emotional data, the interviewer agent dynamically adjusts the tone and content of the questions.

[0722] Step 6:

[0723] Once the interview is over, the server organizes the collected interview data and sentiment data, and the data is summarized by the analysis agent. The input is the collected response data and sentiment data, and the output is specific patterns and business insights.

[0724] Step 7:

[0725] The server displays the analysis results on the user's dashboard, allowing the user to gain specific insights into the effectiveness of their advertising campaigns and areas for improvement. The input is the analyzed data and business insights, and the output is the resulting information displayed on the dashboard.

[0726] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0727] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0728] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0729] [Third embodiment]

[0730] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0731] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0732] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0733] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0734] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0735] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0736] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0737] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0738] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0739] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0740] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0741] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0742] As an embodiment of the present invention, we provide a virtual interview system that utilizes a large-scale language model (LLM) agent. This system is designed according to the following steps:

[0743] 1. Setting interview questions and hypotheses

[0744] The user logs into the system and inputs the purpose of the interview, the tasks, and the desired hypotheses. The server receives this information and saves it as an interview task. This step clarifies the direction the interview requires and the hypotheses to be verified.

[0745] 2. Instructions and implementation of interview content

[0746] The user specifies specific interview questions and answer formats to the server. For example, the user may set questions to check whether there are any points that are difficult to understand when using the app for the first time. The server builds an interview framework based on these questions and sends it to an interviewer agent. The interviewer agent then conducts the interview with the virtual subject and collects the answers.

[0747] 3. Conducting repeated interviews

[0748] The user sets different personas and repeatedly conducts interviews based on each persona. The server generates a virtual subject for each persona and conducts interviews with them to collect diverse data. For example, the same interview can be conducted with persona A (a young working adult) and persona B (an older housewife) to obtain answers from each perspective.

[0749] 4. Summarizing data and extracting business insights

[0750] After all interviews are completed, the server organizes the collected data, which is then summarized by the analyst agent. The analyst agent analyzes the interview results and extracts specific patterns and business insights. The resulting insights and improvement suggestions are displayed on the user's dashboard, allowing them to make real business improvements.

[0751] Specific examples

[0752] Suppose a user wants to evaluate the usability of a new mobile app. The user logs in to the system, sets the interview objective as "evaluating usability," and sets the hypothesis as "the new UI is intuitive to users." Next, the user sets questions to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and asks for free-form responses. Based on this, the server generates an interviewer agent and conducts an interview with the virtual subject.

[0753] The user then sets two personas: a young working person and an elderly housewife. The server repeatedly interviews these personas and stores their responses in a database. Once the interviews are complete, the server uses an analyst agent to analyze the data and discovers that many users report difficulty understanding the navigation system upon first use. Based on this, the analyst agent suggests adding a tutorial guide as an improvement and displays it on the user's dashboard.

[0754] In this way, the system efficiently conducts virtual interviews and provides specific, actionable business insights, thereby improving the efficiency and effectiveness of marketing operations.

[0755] The processing flow will be explained below.

[0756] Step 1:

[0757] A user logs in to the system. The user enters authentication information and accesses the system. The server verifies the user's authentication information, and if authentication is successful, the user is redirected to the dashboard.

[0758] Step 2:

[0759] The user inputs the purpose of the interview and the hypothesis. The user inputs the purpose of the interview (e.g., "Evaluate the usability of a new mobile app") and the hypothesis (e.g., "The new UI is intuitive to users") from the device. The server receives this information and saves it as an interview assignment.

[0760] Step 3:

[0761] The user sets the interview questions and answer format. The user inputs specific questions (e.g., "Which features were difficult to use?") from the terminal and determines the answer format (e.g., free text, multiple choice). The server receives these questions and answer formats and sends them to the interviewer agent.

[0762] Step 4:

[0763] The interviewer agent conducts an interview with the virtual subject. The server creates an interviewer agent and starts interviewing the virtual subject. The interviewer agent presents questions one by one and collects answers from the virtual subject. The server records these answers in real time.

[0764] Step 5:

[0765] Users set different personas. Users set target personas (e.g., young working people, elderly housewives) from their terminals. The server generates virtual targets based on each persona.

[0766] Step 6:

[0767] The server conducts interviews for each persona. The server generates an interviewer agent for each persona and repeatedly conducts interviews with the virtual subject. The answers from this virtual subject are stored in a database.

[0768] Step 7:

[0769] The server summarizes the interview data. After ensuring all interviews are complete, the server sends the collected data to the analyst agent, who then summarizes it.

[0770] Step 8:

[0771] Analyst agents extract business insights. Analyst agents analyze summary data and extract useful insights and improvement suggestions (e.g., discovering the insight that "navigation is difficult to understand on first use").

[0772] Step 9:

[0773] The server displays the insights and improvement suggestions on the user's dashboard. The extracted insights and improvement suggestions are visually displayed on the dashboard for the user to use, allowing the user to take concrete improvement measures based on the interview results.

[0774] This series of steps allows users to conduct interview research efficiently and effectively and improve their business based on the data obtained.

[0775] Example 1

[0776] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0777] With conventional interview systems, it was difficult to design and conduct interviews using virtual subjects, making it difficult to quickly obtain effective business insights. Furthermore, it was often necessary to conduct repeated interviews with different personas, and it was difficult to efficiently analyze the collected data. This resulted in delays in decision-making for marketing and product improvement.

[0778] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0779] In this invention, the server includes: a means for a user to log in and set interview tasks and hypotheses; a means for inputting the purpose, tasks, and expected hypotheses of the interview; a means for an interviewer agent to conduct a virtual interview based on user instructions for the content of the interview; a means for setting different personas and repeatedly conducting interviews based on each persona; a means for the interviewer agent to conduct interviews with virtual subjects and collect responses; a means for saving the collected interview content as data; a means for an analyst agent to organize and analyze the collected data and extract business insights and improvement proposals; and a means for displaying the extracted insights and improvement proposals on the user's dashboard. This enables efficient interview design and implementation using virtual subjects, enabling quick and effective business insights to be obtained.

[0780] "User" refers to the entity that uses the system to design, conduct, and analyze the results of interviews.

[0781] "Server" refers to the computer system that receives input information from users and manages and executes instructions from interviewer agents and analyst agents.

[0782] "Interview questions" refer to the objectives or hypotheses you want to clarify through the interview.

[0783] A "hypothesis" is a prediction or conjecture that you intend to verify through an interview.

[0784] "Interview content" refers to the questions and answer format set during the interview.

[0785] An "interviewer agent" is an artificial intelligence that conducts interviews with virtual subjects and collects their responses.

[0786] "Virtual subject" refers to a virtual interviewee generated using LLM (large-scale language model).

[0787] A "persona" is a fictional character that represents the characteristics and attributes of the person being interviewed.

[0788] "Analyst Agent" refers to an artificial intelligence that organizes and analyzes collected interview results and extracts business insights and improvement proposals.

[0789] A "dashboard" refers to an interface that displays analysis results such as insights and improvement suggestions to users.

[0790] "Business insights" refers to useful business knowledge and insights gained from the interview results.

[0791] "Improvement proposals" refer to specific measures for improving business operations or products that are proposed based on the insights gained.

[0792] The present invention relates to a virtual interview system utilizing a large-scale language model (LLM) agent. This system includes a means for a user to log in and set interview questions and hypotheses, a means for an interviewer agent to conduct virtual interviews based on the interview content specified by the user, a means for repeatedly conducting interviews based on different personas, a means for collecting and analyzing interview results, and a means for displaying the obtained insights and improvement suggestions on the user's dashboard.

[0793] Hardware or software used

[0794] Server: Responsible for setting interview topics and hypotheses, managing and operating interviewer and analyst agents, saving and analyzing interview results, and displaying them on a dashboard. A cloud-based service can be used as a specific server.

[0795] Terminal: The device from which the user accesses the system, such as a PC, tablet, or smartphone.

[0796] Large-scale language models (LLMs): Used to process interviewer and analyst agents. Examples include OpenAI's GPT series.

[0797] Database: Data storage for storing collected interview responses and later analysis.

[0798] Program processing overview

[0799] User login and information entry

[0800] The user logs in to the system and inputs the purpose of the interview, the task, and the desired hypothesis. The server receives this information and saves it as an interview task. For example, suppose the user's goal is to "evaluate the usability of a new mobile app" and sets the hypothesis that "the new UI is intuitive to users."

[0801] Interview content setting and instructions

[0802] The user specifies specific interview questions and answer formats to the server. For example, the server sets questions to check whether there are any points that are difficult to understand when using the app for the first time, and requests free-form answers. The server then sends these to the interview agent and generates a framework for the interview.

[0803] Generate virtual subjects and conduct interviews

[0804] Users can set different personas, for example, a young working person and an elderly housewife. The server uses LLM to generate virtual subjects based on these personas. The interviewer agent then interviews the virtual subjects and collects their responses.

[0805] Collecting and storing interview results

[0806] The server stores the interview results in a database. For example, a young worker may give feedback that the navigation was difficult to understand the first time they used the app.

[0807] Data Summarization and Business Insight Extraction

[0808] After all the interviews are completed, the server organizes the collected data and sends it to the analyst agent, who analyzes the data and extracts specific patterns and business insights. For example, the analyst agent might learn that many users find the navigation difficult to understand the first time they use the service.

[0809] Displaying insights on dashboards

[0810] The server displays the insights obtained from the analyst agent on the user's dashboard. The user can review the insights on the dashboard and consider business improvement measures as needed. For example, a suggested improvement would be to "add a tutorial guide."

[0811] Examples of prompt statements

[0812] "Conduct virtual interviews about the difficulties users experience when using a mobile app for the first time. The target personas are young professionals and older housewives. For each persona, collect specific feedback on the first-time navigation in an open-ended format."

[0813] In this way, the present invention enables efficient virtual interviews and provides specific and practical business insights, thereby improving the efficiency and effectiveness of marketing operations and product improvements.

[0814] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0815] Program processing steps

[0816] Step 1: User login and interview information

[0817] Specific behavior:

[0818] 1. The user accesses the system from a terminal and enters their username and password on the login page.

[0819] 2. The server authenticates the entered username and password and, if authentication is successful, redirects the user to the dashboard.

[0820] 3. On the dashboard, the user clicks the "Create a new interview" button and enters the interview purpose, task, and desired hypotheses.

[0821] Input: Username, Password, Interview Objective, Task, Hypothesis

[0822] Data processing: User authentication, saving interview information

[0823] Output: User login status, input information saved as interview assignment

[0824] Step 2: Interview setup and instructions

[0825] Specific behavior:

[0826] 1. Users enter specific interview questions and answer formats on the dashboard.

[0827] For example: "Please tell us about any difficulties you encountered when using the app for the first time."

[0828] 2. The server saves the entered questions and prepares instructions for the interviewer agent.

[0829] Input: Specific question and answer format

[0830] Data processing: saving questions and preparing instructions

[0831] Output: Information saved as interview questions, prepared instructions for the interviewer agent

[0832] Step 3: Generate a virtual subject and conduct interviews

[0833] Specific behavior:

[0834] 1. Users can create different personas on the dashboard.

[0835] Examples: young working people, older housewives.

[0836] 2. The server uses LLM to generate virtual targets based on the configured personas.

[0837] 3. The interviewer agent asks the generated virtual subject questions set by the user.

[0838] Input: Persona settings, questions

[0839] Data processing: Creating virtual subjects and conducting questions and answers

[0840] Output: Generated virtual audience, collected responses

[0841] Step 4: Collect and store interview results

[0842] Specific behavior:

[0843] 1. The interviewer agent sends the answers obtained from the virtual subject to the server.

[0844] 2. The server stores the interview results in a database.

[0845] Example: Feedback from young working adults: "The navigation is difficult to understand when using it for the first time."

[0846] Input: Responses from hypothetical subjects

[0847] Data processing: Saving answers

[0848] Output: Archived interview results data

[0849] Step 5: Summarize the data and extract business insights

[0850] Specific behavior:

[0851] 1. The server organizes the interview results stored in the database and sends them to the analyst agent.

[0852] 2. Analyst agents analyze the data and extract specific patterns and business insights.

[0853] Example: "Many users find the navigation difficult to understand on first use."

[0854] Input: Saved interview result data

[0855] Data processing: Data organization and analysis

[0856] Output: Extracted business insights and improvement suggestions

[0857] Step 6: Display insights in a dashboard

[0858] Specific behavior:

[0859] 1. The server displays insights and improvement suggestions obtained from the analyst agent on the user's dashboard.

[0860] 2. Users review insights on the dashboard and consider business improvement measures as needed.

[0861] Example: Consider adding a tutorial guide.

[0862] Input: Insights and improvement suggestions from analyst agents

[0863] Data processing: Displaying insights and improvement suggestions

[0864] Output: Insights and recommendations displayed on the user's dashboard

[0865] By clearly showing the specific operations and inputs / outputs at each processing step, it is possible to explain the entire processing flow of the system in detail, allowing users to efficiently conduct virtual interviews and obtain specific, actionable business insights.

[0866] (Application example 1)

[0867] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0868] In modern businesses, efficiently collecting customer feedback is essential for improving products and services. However, traditional methods have made it difficult to accurately collect and analyze real-time customer feedback in physical stores. Furthermore, there has been a lack of appropriate tools to instantly analyze collected feedback and utilize it as business insights. An effective system is needed to resolve these issues and improve the quality of store operations.

[0869] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0870] In this invention, the server includes means for setting interview topics and hypotheses, means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on that content, means for repeating and additionally conducting interviews, means for analyzing the obtained interview content and extracting business insights and improvement proposals, means for displaying the interview content on smart glasses for collecting feedback from customers in physical stores and recording it as audio, and means for converting the audio data into text and transmitting it to a cloud server. This makes it possible to efficiently collect and analyze real-time customer feedback in physical stores and quickly obtain business insights.

[0871] An "interview question" is the purpose and content of questions set based on the information you want to collect and the hypothesis you want to verify in the virtual interview.

[0872] A "hypothesis" is a premise or condition that is tested by an interview and is confirmed or denied through a series of interviews.

[0873] "Interview content" refers to detailed instructions including specific questions and answer formats that the user will ask in the interview.

[0874] A "user" is someone who uses the interview system to set interview questions and hypotheses, instruct the content of the interview, and analyze the collected data.

[0875] An "interviewer agent" is an agent that conducts an interview based on the interview content set for a virtual subject.

[0876] "Virtual subjects" are hypothetical respondents generated for interviews and based on different personas.

[0877] A "persona" is a fictional character with different attributes and backgrounds that represents a specific target user.

[0878] "Smart glasses" are wearable devices that have the ability to display specific information and record audio data in real time.

[0879] A "cloud server" is a remote server that stores, processes, and analyzes data and is accessed over a network.

[0880] "Voice data" refers to voice response information collected by users or store staff through smart glasses.

[0881] "Convert to text" refers to the process of converting collected voice data into text data.

[0882] "Business insights" are insights and findings that are useful for business improvement and decision-making, based on the analysis of collected data.

[0883] The system includes a means for setting interview topics and hypotheses, a means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on that content, a means for conducting repeated and additional interviews, a means for analyzing the obtained interview content and extracting business insights and improvement proposals, a means for displaying the interview content on smart glasses for collecting feedback from customers in physical stores and recording it as audio, and a means for converting the audio data into text and sending it to a cloud server.

[0884] Specific system implementation includes the following elements:

[0885] 1. Hardware and Software Used:

[0886] Smart glasses: A device used to display the interview content and record audio data.

[0887] Cloud server: A remote server (e.g., AWS) that stores, processes, and analyzes data.

[0888] Speech recognition software: Software that converts voice data into text data (e.g., Google Cloud Speech-to-Text API).

[0889] Large-scale language models: Software used to analyze interview results and extract insights (e.g., OpenAI GPT-4).

[0890] Database: A database (e.g. MySQL) to store the collected data.

[0891] 2. System operation process:

[0892] The server first sets the interview topic and hypotheses. Based on this, the user specifies the interview content, and the interviewer agent conducts the interview with the virtual subjects. The virtual subjects are generated based on personas, and different interviews are repeatedly conducted for each persona.

[0893] In a physical store, the customer wears smart glasses, and the interview content is displayed on the glasses. The customer asks questions to the customer, which are recorded as audio data. This audio data is converted into text in real time and sent to a cloud server.

[0894] The cloud server stores the submitted text data and analyzes it using large-scale language models to extract specific patterns and business insights, which are then displayed on the user's dashboard.

[0895] 3. Example:

[0896] Suppose you want to investigate which aspects of a new store layout interest customers and which aspects dissatisfy them after the new layout is implemented. To do this, set the following hypotheses and questions:

[0897] Hypothesis: "The new layout will make customer flow more smoothly."

[0898] question:

[0899] 1. What did you think of the new layout?

[0900] 2. What area of ​​the store appealed to you the most?

[0901] 3. Were there any aspects that you were dissatisfied with?

[0902] Example prompt sentence:

[0903] Customer Feedback Data:

[0904] 1. What did you think of the new layout?

[0905] "The flow of traffic was good and it was easy to shop." "I got a little lost."

[0906] 2. What area of ​​the store appealed to you the most?

[0907] "Fashion corner" "Food corner"

[0908] 3. Were there any aspects that you were dissatisfied with?

[0909] "The area around the cash register was crowded" "The parking lot was small"

[0910] Based on the customer feedback below, summarize the key themes and insights.

[0911] This system efficiently collects customer feedback in physical stores and instantly analyzes the data, enabling quick and specific business insights.

[0912] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0913] Step 1:

[0914] Users log in to the system and set interview questions and hypotheses. These are sent to the cloud server and saved as interview questions. This process clarifies the purpose and direction of the interview.

[0915] Input: Interview objectives, questions, hypotheses

[0916] Output: Saved interview assignments

[0917] Specific operation: Receives user input data, sends it to the cloud server, and stores it in the database.

[0918] Step 2:

[0919] The user provides specific interview questions and answer formats to the cloud server, which then builds a framework for an interviewer agent based on this information, and the interviewer agent then conducts an interview with the virtual subject.

[0920] Input: Interview questions, answers

[0921] Output: Constructed interview framework, collected response data

[0922] Specific operations: Receives user instructions, generates a set of questions for the interviewer agent, and conducts an interview with the virtual subject.

[0923] Step 3:

[0924] The cloud server generates different personas and conducts repeated interviews with virtual subjects based on each persona, thereby collecting feedback from a variety of perspectives.

[0925] Input: Persona information

[0926] Output: A set of response data from each persona

[0927] What it does: Generate multiple personas and gather relevant data by conducting specialized virtual interviews for each.

[0928] Step 4:

[0929] The user wears the smart glasses, and the interview content is displayed on the glasses. Questions are asked to the customer, and their answers are recorded as audio.

[0930] Input: Interview Question

[0931] Output: Recorded audio data

[0932] Specific operation: The interview content is displayed on the smart glasses and the customer's voice responses are recorded.

[0933] Step 5:

[0934] The voice data is converted into text in real time and sent to a cloud server using voice recognition software.

[0935] Input: Audio data

[0936] Output: Text data

[0937] What it does: It uses speech recognition software to convert voice data into text and sends it to a cloud server.

[0938] Step 6:

[0939] The cloud server stores the transmitted text data in a database and analyzes the data using a generative AI model.

[0940] Input: Text data

[0941] Output: Analysis results, business insights

[0942] What it does: Collected text data is stored in a database and patterns and insights are extracted using generative AI models.

[0943] Step 7:

[0944] The server displays the insights and improvement suggestions obtained in real time on the user's dashboard.

[0945] Input: Analysis results

[0946] Output: Business insight dashboard display

[0947] What it does: Updates user dashboards with analysis results, providing real-time business insights.

[0948] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0949] As an embodiment of the present invention, we provide a virtual interview system that utilizes a large-scale language model (LLM) agent and an emotion engine. This system is designed according to the following steps:

[0950] 1. Setting interview questions and hypotheses

[0951] The user logs into the system and inputs the purpose of the interview, the tasks, and the desired hypotheses. The server receives this information and saves it as an interview task. This step clarifies the direction the interview requires and the hypotheses to be verified.

[0952] 2. Instructions and implementation of interview content

[0953] The user specifies specific interview questions and answer formats to the server. For example, the user may set questions to check whether there are any points that are difficult to understand when using the app for the first time. The server builds an interview framework based on these questions and sends it to an interviewer agent. The interviewer agent then conducts the interview with the virtual subject and collects the answers.

[0954] 3. Conducting repeated interviews

[0955] The user sets different personas and repeatedly conducts interviews based on each persona. The server generates a virtual subject for each persona and conducts interviews with them to collect diverse data. For example, the same interview can be conducted with persona A (a young working adult) and persona B (an older housewife) to obtain answers from each perspective.

[0956] 4. Applying the Emotion Engine

[0957] During the interview, the server utilizes an emotion engine to recognize the virtual subject's emotional data in real time. The emotion engine detects the subject's emotional state from their voice tone, choice of words, and reaction speed. This allows the interviewer agent to dynamically adjust the tone and content of questions according to the virtual subject's emotions.

[0958] 5. Summarizing data and extracting business insights

[0959] After all interviews are completed, the server organizes the collected data, and the analyst agent summarizes it. The analyst agent integrates the interview results with sentiment data to extract specific patterns and business insights. The resulting insights and improvement suggestions are displayed on the user's dashboard, allowing them to make real business improvements.

[0960] Specific examples

[0961] Suppose a user wants to evaluate the usability of a new mobile app. The user logs in to the system, sets the interview objective as "evaluating usability," and sets the hypothesis as "the new UI is intuitive to users." Next, the user sets questions to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and asks for free-form responses. Based on this, the server generates an interviewer agent and conducts an interview with the virtual subject.

[0962] The user then sets up two personas: a young working professional and an elderly housewife. The server repeatedly interviews these personas and stores their responses in a database. During the interview, the emotion engine recognizes the virtual subjects' emotions in real time, allowing the interviewer agent to adjust the questions they ask based on the emotional data, thereby gaining deeper insights.

[0963] Once the interviews are complete, the server uses an analyst agent to analyze the data and discover that many users report difficulty with navigation during first-time use. Based on this feedback, including sentiment data, the analyst agent suggests adding a tutorial guide as an improvement and displays it on the user's dashboard.

[0964] In this way, the system efficiently combines virtual interviews and emotion recognition to provide specific, actionable business insights, thereby improving the efficiency and effectiveness of marketing operations.

[0965] The processing flow will be explained below.

[0966] Step 1:

[0967] A user logs in to the system. The user enters their authentication information and accesses the system. The server verifies the user's authentication information, and if authentication is successful, the user is redirected to the user's dashboard.

[0968] Step 2:

[0969] The user inputs the purpose of the interview and the hypothesis from the terminal. The user inputs the purpose of the interview (e.g., "Evaluate the usability of a new mobile app") and the hypothesis (e.g., "The new UI is intuitive to users"). The server receives this information and saves it as an interview assignment.

[0970] Step 3:

[0971] The user sets the interview questions and answer format from the terminal. The user inputs specific questions (e.g., "Which features were difficult to use?") and answer formats (e.g., free text, multiple choice). The server receives these questions and builds the framework for the interview.

[0972] Step 4:

[0973] The server generates an interviewer agent to conduct an interview with the virtual subject. The interviewer agent asks questions to the virtual subject based on the questions received from the server and collects answers. The server records these answers in real time.

[0974] Step 5:

[0975] The user sets different personas from the terminal. The user sets the target persona (e.g., young working people, elderly housewives). The server generates virtual targets based on each persona.

[0976] Step 6:

[0977] The server conducts interviews for each persona. The server generates an interviewer agent for each persona and repeatedly conducts interviews with the virtual subject. The answers from this virtual subject are stored in a database.

[0978] Step 7:

[0979] The server uses an emotion engine to recognize the virtual subject's emotions in real time. The emotion engine detects the subject's emotional state from their tone of voice, choice of words, and reaction speed. During the interview, the emotional data is fed back to the interviewer agent, who dynamically adjusts the tone and content of the questions.

[0980] Step 8:

[0981] The server summarizes the interview data. After ensuring all interviews are complete, the server sends the collected data to the analyst agent, who then summarizes it.

[0982] Step 9:

[0983] Analyst agents extract business insights. Analyst agents analyze summary data and extract useful insights and improvement suggestions (e.g., discovering that "navigation is difficult to understand on first use"). This process also integrates sentiment data for more accurate analysis.

[0984] Step 10:

[0985] The server displays the insights and improvement suggestions on the user's dashboard. The extracted insights and improvement suggestions are visually displayed on the dashboard for the user to use, allowing the user to take concrete improvement measures based on the interview results and sentiment data.

[0986] This series of steps allows users to conduct interview surveys efficiently and effectively, and improve their business based on the obtained data and emotion recognition information.

[0987] Example 2

[0988] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0989] Conventional interview systems are unable to capture the subject's emotions in real time, resulting in less nuanced and reliable answers. They also struggle to collect data from diverse perspectives based on different personas, leading to insufficient extraction of business insights. Furthermore, it is difficult to efficiently analyze the collected data and provide users with immediate feedback and improvement suggestions based on that data.

[0990] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0991] In this invention, the server includes: means for a user to set interview topics and hypotheses; means for a user to specify interview content and for an interviewer agent to conduct interviews with virtual subjects based on the content; means for conducting repeated and additional interviews; means for utilizing an emotion engine to recognize emotional data of the virtual subjects in real time during the interview and dynamically adjust the content and tone of questions; means for generating multiple different personas and conducting interviews with these personas to collect data; means for recording interview results and emotional data in real time and displaying them on a dashboard on a user's operating terminal; and means for analyzing the obtained interview content and emotional data and extracting business insights and improvement proposals. This makes it possible to recognize the emotions of virtual subjects in real time, collect data from various personas, and provide efficient feedback and business insights.

[0992] The "interview topic" refers to the purpose and theme of the interview, and the hypothesis you want to verify.

[0993] A "virtual subject" is a simulated interviewee generated by a computer system.

[0994] An "interviewer agent" is a program or system that conducts interviews with virtual subjects, posing questions and collecting responses.

[0995] The "emotion engine" is a technology that detects emotional states from voice tone, selected words, reaction speed, etc., and recognizes emotional data in real time.

[0996] A "persona" is a fictional character with specific attributes and behavioral patterns, and is used as a representative model for marketing and product design.

[0997] "Business insights" are insights and suggestions that are useful for specific business improvements and decision-making, obtained by analyzing collected data.

[0998] A "dashboard" is an interface that visually displays collected data and analysis results, allowing users to easily view and operate them.

[0999] A "generative AI model" refers to an artificial intelligence model generated by a machine learning algorithm using a large dataset.

[1000] A "prompt" is an instruction used to ask a specific question or give input to an AI model.

[1001] "Response format" refers to the format in which responses to interview questions are collected, and includes free-form and multiple-choice formats.

[1002] This invention is a virtual interview system that utilizes a large-scale language model (LLM) agent and an emotion engine, which allows users to obtain emotion data of virtual subjects in real time and efficiently conduct interviews based on various personas.

[1003] System Configuration

[1004] The system consists of three elements: a server, a terminal, and a user, and operates as follows.

[1005] 1. User Roles

[1006] The user logs in to the system using a terminal and sets the purpose, task, and hypothesis of the interview, which instructs the interviewer agent on the interview content and sends the interview task to the server.

[1007] 2. Server Roles

[1008] The server receives the interview objectives and hypotheses sent by the user and stores them in a database as interview tasks. Next, it generates an interviewer agent and prepares to conduct an interview with the virtual subject. To do this, it uses a generative AI model. The server utilizes an emotion engine to recognize the virtual subject's emotional data in real time and dynamically adjust the content and tone of the questions. The obtained interview content and emotional data are analyzed by the server and provided to the user as business insights and improvement suggestions.

[1009] 3. Roles of the Interviewer Agent and Emotion Engine

[1010] The interviewer agent conducts interviews with virtual subjects and collects their responses. It also analyzes the virtual subjects' emotional data in real time through an emotion engine, dynamically adjusting the content and tone of the interview. The emotion engine detects emotions from voice tone, choice of words, and reaction speed.

[1011] Specific steps and operations

[1012] 1. User login and interview preparation

[1013] The user logs in to the system from their device and sets the purpose of the interview as "evaluating usability" and the hypothesis as "the new UI is intuitive to users." Next, questions are set to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and responses are asked in the form of free text.

[1014] 2. Create personas and conduct interviews

[1015] The user sets two personas: a young working person and an elderly housewife. The server generates virtual subjects based on these personas, operates an interviewer agent, and repeatedly conducts the same interview with each of them. During this process, the emotion engine recognizes the virtual subjects' emotional data in real time, and the interviewer agent adjusts the questions accordingly.

[1016] 3. Analyzing data and providing business insights

[1017] After the interview is completed, the server sends the collected data to an analyst agent for analysis. The analyst agent integrates the interview results with sentiment data to extract specific patterns and insights. For example, the analyst agent may discover that many users report difficulty with navigation during first-time use and suggest adding a tutorial guide. These insights and improvement suggestions are displayed on the user's dashboard.

[1018] Examples of concrete examples and prompts

[1019] As a concrete example, the following prompt sentence can be used:

[1020] Example prompt sentence:

[1021] "We would like to conduct interviews to evaluate the usability of a new mobile app. The purpose of the interview is to 'evaluate usability', and the hypothesis is that 'the new UI is intuitive to users.' We will set questions to identify any aspects of how to use the app that are difficult to understand upon first use, and will ask for free-form responses. The target personas will be young working people and older housewives."

[1022] With these specific operating procedures and system configurations, the present invention can realize efficient and accurate virtual interviews and the extraction of various business insights.

[1023] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1024] Step 1:

[1025] The user logs into the system from a terminal. The user's ID and password are required as input. The login information is sent to the server, which then references the database for authentication. If this authentication is successful, the user is taken to a screen where they can set the interview topic and hypothesis. Specifically, the user enters their ID and password into the login screen and clicks the "Login" button.

[1026] Step 2:

[1027] The user inputs the purpose, task, and hypothesis of the interview and sends it to the server. The input requires the purpose, task, and hypothesis of the interview. The server receives this information and stores it in the database as an interview task. Specifically, the user inputs the purpose and hypothesis of the interview into the form and presses the "Submit" button.

[1028] Step 3:

[1029] The user specifies the interview content and answer format. The question content and answer format are required as input. The server generates an interview framework based on this and builds an interviewer agent. Specifically, the user inputs the question content and answer format and presses the "Send" button. The server generates a question template using a generative AI model.

[1030] Step 4:

[1031] The server launches an interviewer agent and conducts an interview with the virtual subject. The inputs required are an interview framework and a profile of the virtual subject. The server instructs the interviewer agent on the content of the interview, and the agent asks questions to the virtual subject. Specifically, the server launches the agent and has it begin asking questions to the virtual subject.

[1032] Step 5:

[1033] The server utilizes an emotion engine to recognize the virtual subject's emotional data in real time. Inputs include the virtual subject's tone of voice, selected words, and reaction speed. The server analyzes this data with the emotion engine to detect their emotional state. Specifically, the server integrates the emotion engine into the interviewer agent and collects emotional data during the interview.

[1034] Step 6:

[1035] The interviewer agent dynamically adjusts the content and tone of questions based on the emotional data of the virtual subject. The emotional data and the current question are required as input. The server modifies the question based on the emotional data and sends instructions to the interviewer agent. Specifically, the server dynamically changes the next question using the results of the emotional data analysis.

[1036] Step 7:

[1037] The user sets up different personas. The persona's attributes (age, occupation, gender, etc.) are required as input. The server generates multiple virtual subjects based on this and conducts repeated interviews with each of them. Specifically, the user enters the persona information and presses the "Submit" button.

[1038] Step 8:

[1039] The server analyzes the collected interview content and sentiment data. The collected interview data and sentiment data are required as input. The server uses analyst agents to analyze the data and generate business insights and improvement proposals. Specifically, the server sends the data to the analyst agents and displays the analysis results on a dashboard.

[1040] Step 9:

[1041] Users can view analysis results on the dashboard and check improvement proposals. Analysis result data is required as input. Business insights and improvement proposals are displayed as output. In concrete terms, users access the dashboard and check the analysis results.

[1042] (Application example 2)

[1043] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1044] Conventional methods for evaluating advertising campaigns have made it difficult to quickly and efficiently collect target audience emotional responses and specific feedback. Furthermore, evaluating the effectiveness of advertising campaigns individually for different personas requires a significant amount of time and effort. The present invention aims to solve these problems by efficiently evaluating advertising campaigns and quickly obtaining specific insights that take into account the target audience's emotional responses.

[1045] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1046] In this invention, the server includes means for setting interview topics and hypotheses, means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content, means for setting a plurality of different personas and repeatedly conducting interviews based on each persona, means for using an emotion engine to recognize emotional data of the virtual subject in real time during the interview and for the interviewer agent to dynamically adjust the tone and content of questions based on the emotion, and means for analyzing the obtained interview content and extracting business insights and improvement proposals. This makes it possible to efficiently evaluate the effectiveness of an advertising campaign while taking into account the emotional responses of the target audience and quickly obtain specific feedback and business insights.

[1047] "Means for setting interview questions and hypotheses" is a function that allows system users to specifically input the objectives and goals of the advertising campaign they wish to evaluate.

[1048] "A means for the user to specify the interview content and for the interviewer agent to conduct a virtual interview based on that" is a function that conducts an interview with a virtual subject based on the questions specified by the user.

[1049] "A means of setting up multiple different personas and repeatedly conducting interviews based on each persona" is a function that virtually sets up different target groups and conducts interviews with each of them to collect data.

[1050] "Using an emotion engine to recognize the emotional data of virtual subjects in real time during an interview, allowing the interviewer agent to dynamically adjust the tone and content of questions based on their emotions" is a function that grasps the emotional state of the subject during the interview and flexibly changes the interview approach accordingly.

[1051] "Means of analyzing the interview content obtained and extracting business insights and improvement proposals" is a function that analyzes the collected interview data and derives specific insights and improvement proposals regarding the effectiveness of advertising campaigns.

[1052] The present invention relates to a system for evaluating the effectiveness of an advertising campaign, which operates in the following manner.

[1053] First, users log in to the system and set the objectives and hypotheses for their advertising campaign. For example, they can set a hypothesis that "the new advertisement will be very popular with young people." This setting information is then stored on the server.

[1054] Next, the user specifies the specific content of the interview. Questions can be set such as "What was your first impression when you saw this advertisement?" or "What emotions did you feel when you saw this advertisement?" and responses can be requested in the form of free description or multiple choice. Based on this, the server generates an interviewer agent and begins interviewing the virtual subject.

[1055] In this process, multiple personas are created to represent different target demographics (for example, "male high school students" and "female university students"), and interviews are conducted repeatedly. This process makes it possible to collect a wide variety of data.

[1056] During the interview, the server uses an emotion engine to recognize the virtual subject's emotional data in real time. The emotion engine detects the subject's emotional state from their voice tone, choice of words, reaction speed, etc. Based on the recognized emotional data, the interviewer agent can dynamically adjust the tone and content of their questions to extract deeper insights.

[1057] Once the interviews are complete, the server organizes the collected data, which is then summarized by an analysis agent. The analysis agent then integrates the interview results with sentiment data to extract specific patterns and business insights. The resulting insights and recommendations are displayed on the user's dashboard, allowing them to use them to improve their actual advertising campaigns.

[1058] This invention allows for efficient evaluation of advertising campaigns that take into account the emotional responses of the target audience. For example, it is possible to quickly obtain specific feedback such as how high school boys felt when they saw the advertisement, or how college girls felt when they saw the advertisement.

[1059] Example prompt sentence:

[1060] "How do high school boys feel when they see this ad?"

[1061] "What emotions did college girls have when they saw this ad?"

[1062] Thus, the present invention is a system that utilizes a large-scale language model and an emotion engine to enable the effectiveness of advertising campaigns to be obtained from virtual audiences that are close to the actual target demographic, thereby maximizing the effectiveness of advertising and providing important insights for formulating more effective advertising strategies for the target demographic.

[1063] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1064] Step 1:

[1065] Users log in to the system from their devices and set the objectives and hypotheses for their advertising campaign. Hypotheses such as "the new advertisement will be very popular with young people" are entered as input, and the server receives and stores this information. Based on this configuration information, the server processes the data to determine the direction of the interviews.

[1066] Step 2:

[1067] The user inputs specific interview content from the terminal. For example, they input questions such as, "What was your first impression when you saw this advertisement?" or "What emotions did you feel when you saw this advertisement?" The server receives these and performs data calculations to pass instructions to the interviewer agent. As a result, a question framework for the interviewer agent is constructed.

[1068] Step 3:

[1069] Users can set different target demographics (personas) on their devices. For example, they can input personas such as "high school boys" or "university girls." The server receives this data and processes it to generate multiple virtual targets based on the specified personas.

[1070] Step 4:

[1071] The interviewer agent conducts interviews with virtual subjects based on instructions from the server. When questions are presented to the subjects, their responses are collected. The input is the question and the virtual subject's prompt, and the output is the collected response data.

[1072] Step 5:

[1073] While the virtual subject is answering, the server uses an emotion engine to recognize and analyze emotional data from the answers in real time. The input is the collected answer data and data on the virtual subject's reactions, and the output is the detected emotional data. Based on this emotional data, the interviewer agent dynamically adjusts the tone and content of the questions.

[1074] Step 6:

[1075] Once the interview is over, the server organizes the collected interview data and sentiment data, and the data is summarized by the analysis agent. The input is the collected response data and sentiment data, and the output is specific patterns and business insights.

[1076] Step 7:

[1077] The server displays the analysis results on the user's dashboard, allowing the user to gain specific insights into the effectiveness of their advertising campaigns and areas for improvement. The input is the analyzed data and business insights, and the output is the resulting information displayed on the dashboard.

[1078] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1079] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1080] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1081] [Fourth embodiment]

[1082] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1083] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1084] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1085] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1086] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1087] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1088] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1089] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1090] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1091] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1092] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1093] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1094] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1095] As an embodiment of the present invention, we provide a virtual interview system that utilizes a large-scale language model (LLM) agent. This system is designed according to the following steps:

[1096] 1. Setting interview questions and hypotheses

[1097] The user logs into the system and inputs the purpose of the interview, the tasks, and the desired hypotheses. The server receives this information and saves it as an interview task. This step clarifies the direction the interview requires and the hypotheses to be verified.

[1098] 2. Instructions and implementation of interview content

[1099] The user specifies specific interview questions and answer formats to the server. For example, the user may set questions to check whether there are any points that are difficult to understand when using the app for the first time. The server builds an interview framework based on these questions and sends it to an interviewer agent. The interviewer agent then conducts the interview with the virtual subject and collects the answers.

[1100] 3. Conducting repeated interviews

[1101] The user sets different personas and repeatedly conducts interviews based on each persona. The server generates a virtual subject for each persona and conducts interviews with them to collect diverse data. For example, the same interview can be conducted with persona A (a young working adult) and persona B (an older housewife) to obtain answers from each perspective.

[1102] 4. Summarizing data and extracting business insights

[1103] After all interviews are completed, the server organizes the collected data, which is then summarized by the analyst agent. The analyst agent analyzes the interview results and extracts specific patterns and business insights. The resulting insights and improvement suggestions are displayed on the user's dashboard, allowing them to make real business improvements.

[1104] Specific examples

[1105] Suppose a user wants to evaluate the usability of a new mobile app. The user logs in to the system, sets the interview objective as "evaluating usability," and sets the hypothesis as "the new UI is intuitive to users." Next, the user sets questions to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and asks for free-form responses. Based on this, the server generates an interviewer agent and conducts an interview with the virtual subject.

[1106] The user then sets two personas: a young working person and an elderly housewife. The server repeatedly interviews these personas and stores their responses in a database. Once the interviews are complete, the server uses an analyst agent to analyze the data and discovers that many users report difficulty understanding the navigation system upon first use. Based on this, the analyst agent suggests adding a tutorial guide as an improvement and displays it on the user's dashboard.

[1107] In this way, the system efficiently conducts virtual interviews and provides specific, actionable business insights, thereby improving the efficiency and effectiveness of marketing operations.

[1108] The processing flow will be explained below.

[1109] Step 1:

[1110] A user logs in to the system. The user enters authentication information and accesses the system. The server verifies the user's authentication information, and if authentication is successful, the user is redirected to the dashboard.

[1111] Step 2:

[1112] The user inputs the purpose of the interview and the hypothesis. The user inputs the purpose of the interview (e.g., "Evaluate the usability of a new mobile app") and the hypothesis (e.g., "The new UI is intuitive to users") from the device. The server receives this information and saves it as an interview assignment.

[1113] Step 3:

[1114] The user sets the interview questions and answer format. The user inputs specific questions (e.g., "Which features were difficult to use?") from the terminal and determines the answer format (e.g., free text, multiple choice). The server receives these questions and answer formats and sends them to the interviewer agent.

[1115] Step 4:

[1116] The interviewer agent conducts an interview with the virtual subject. The server creates an interviewer agent and starts interviewing the virtual subject. The interviewer agent presents questions one by one and collects answers from the virtual subject. The server records these answers in real time.

[1117] Step 5:

[1118] Users set different personas. Users set target personas (e.g., young working people, elderly housewives) from their terminals. The server generates virtual targets based on each persona.

[1119] Step 6:

[1120] The server conducts interviews for each persona. The server generates an interviewer agent for each persona and repeatedly conducts interviews with the virtual subject. The answers from this virtual subject are stored in a database.

[1121] Step 7:

[1122] The server summarizes the interview data. After ensuring all interviews are complete, the server sends the collected data to the analyst agent, who then summarizes it.

[1123] Step 8:

[1124] Analyst agents extract business insights. Analyst agents analyze summary data and extract useful insights and improvement suggestions (e.g., discovering the insight that "navigation is difficult to understand on first use").

[1125] Step 9:

[1126] The server displays the insights and improvement suggestions on the user's dashboard. The extracted insights and improvement suggestions are visually displayed on the dashboard for the user to use, allowing the user to take concrete improvement measures based on the interview results.

[1127] This series of steps allows users to conduct interview research efficiently and effectively and improve their business based on the data obtained.

[1128] Example 1

[1129] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1130] With conventional interview systems, it was difficult to design and conduct interviews using virtual subjects, making it difficult to quickly obtain effective business insights. Furthermore, it was often necessary to conduct repeated interviews with different personas, and it was difficult to efficiently analyze the collected data. This resulted in delays in decision-making for marketing and product improvement.

[1131] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1132] In this invention, the server includes: a means for a user to log in and set interview tasks and hypotheses; a means for inputting the purpose, tasks, and expected hypotheses of the interview; a means for an interviewer agent to conduct a virtual interview based on user instructions for the content of the interview; a means for setting different personas and repeatedly conducting interviews based on each persona; a means for the interviewer agent to conduct interviews with virtual subjects and collect responses; a means for saving the collected interview content as data; a means for an analyst agent to organize and analyze the collected data and extract business insights and improvement proposals; and a means for displaying the extracted insights and improvement proposals on the user's dashboard. This enables efficient interview design and implementation using virtual subjects, enabling quick and effective business insights to be obtained.

[1133] "User" refers to the entity that uses the system to design, conduct, and analyze the results of interviews.

[1134] "Server" refers to the computer system that receives input information from users and manages and executes instructions from interviewer agents and analyst agents.

[1135] "Interview questions" refer to the objectives or hypotheses you want to clarify through the interview.

[1136] A "hypothesis" is a prediction or conjecture that you intend to verify through an interview.

[1137] "Interview content" refers to the questions and answer format set during the interview.

[1138] An "interviewer agent" is an artificial intelligence that conducts interviews with virtual subjects and collects their responses.

[1139] "Virtual subject" refers to a virtual interviewee generated using LLM (large-scale language model).

[1140] A "persona" is a fictional character that represents the characteristics and attributes of the person being interviewed.

[1141] "Analyst Agent" refers to an artificial intelligence that organizes and analyzes collected interview results and extracts business insights and improvement proposals.

[1142] A "dashboard" refers to an interface that displays analysis results such as insights and improvement suggestions to users.

[1143] "Business insights" refers to useful business knowledge and insights gained from the interview results.

[1144] "Improvement proposals" refer to specific measures for improving business operations or products that are proposed based on the insights gained.

[1145] The present invention relates to a virtual interview system utilizing a large-scale language model (LLM) agent. This system includes a means for a user to log in and set interview questions and hypotheses, a means for an interviewer agent to conduct virtual interviews based on the interview content specified by the user, a means for repeatedly conducting interviews based on different personas, a means for collecting and analyzing interview results, and a means for displaying the obtained insights and improvement suggestions on the user's dashboard.

[1146] Hardware or software used

[1147] Server: Responsible for setting interview topics and hypotheses, managing and operating interviewer and analyst agents, saving and analyzing interview results, and displaying them on a dashboard. A cloud-based service can be used as a specific server.

[1148] Terminal: The device from which the user accesses the system, such as a PC, tablet, or smartphone.

[1149] Large-scale language models (LLMs): Used to process interviewer and analyst agents. Examples include OpenAI's GPT series.

[1150] Database: Data storage for storing collected interview responses and later analysis.

[1151] Program processing overview

[1152] User login and information entry

[1153] The user logs in to the system and inputs the purpose of the interview, the task, and the desired hypothesis. The server receives this information and saves it as an interview task. For example, suppose the user's goal is to "evaluate the usability of a new mobile app" and sets the hypothesis that "the new UI is intuitive to users."

[1154] Interview content setting and instructions

[1155] The user specifies specific interview questions and answer formats to the server. For example, the server sets questions to check whether there are any points that are difficult to understand when using the app for the first time, and requests free-form answers. The server then sends these to the interview agent and generates a framework for the interview.

[1156] Generate virtual subjects and conduct interviews

[1157] Users can set different personas, for example, a young working person and an elderly housewife. The server uses LLM to generate virtual subjects based on these personas. The interviewer agent then interviews the virtual subjects and collects their responses.

[1158] Collecting and storing interview results

[1159] The server stores the interview results in a database. For example, a young worker may give feedback that the navigation was difficult to understand the first time they used the app.

[1160] Data Summarization and Business Insight Extraction

[1161] After all the interviews are completed, the server organizes the collected data and sends it to the analyst agent, who analyzes the data and extracts specific patterns and business insights. For example, the analyst agent might learn that many users find the navigation difficult to understand the first time they use the service.

[1162] Displaying insights on dashboards

[1163] The server displays the insights obtained from the analyst agent on the user's dashboard. The user can review the insights on the dashboard and consider business improvement measures as needed. For example, a suggested improvement would be to "add a tutorial guide."

[1164] Examples of prompt statements

[1165] "Conduct virtual interviews about the difficulties users experience when using a mobile app for the first time. The target personas are young professionals and older housewives. For each persona, collect specific feedback on the first-time navigation in an open-ended format."

[1166] In this way, the present invention enables efficient virtual interviews and provides specific and practical business insights, thereby improving the efficiency and effectiveness of marketing operations and product improvements.

[1167] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1168] Program processing steps

[1169] Step 1: User login and interview information

[1170] Specific behavior:

[1171] 1. The user accesses the system from a terminal and enters their username and password on the login page.

[1172] 2. The server authenticates the entered username and password and, if authentication is successful, redirects the user to the dashboard.

[1173] 3. On the dashboard, the user clicks the "Create a new interview" button and enters the interview purpose, task, and desired hypotheses.

[1174] Input: Username, Password, Interview Objective, Task, Hypothesis

[1175] Data processing: User authentication, saving interview information

[1176] Output: User login status, input information saved as interview assignment

[1177] Step 2: Interview setup and instructions

[1178] Specific behavior:

[1179] 1. Users enter specific interview questions and answer formats on the dashboard.

[1180] For example: "Please tell us about any difficulties you encountered when using the app for the first time."

[1181] 2. The server saves the entered questions and prepares instructions for the interviewer agent.

[1182] Input: Specific question and answer format

[1183] Data processing: saving questions and preparing instructions

[1184] Output: Information saved as interview questions, prepared instructions for the interviewer agent

[1185] Step 3: Generate a virtual subject and conduct interviews

[1186] Specific behavior:

[1187] 1. Users can create different personas on the dashboard.

[1188] Examples: young working people, older housewives.

[1189] 2. The server uses LLM to generate virtual targets based on the configured personas.

[1190] 3. The interviewer agent asks the generated virtual subject questions set by the user.

[1191] Input: Persona settings, questions

[1192] Data processing: Creating virtual subjects and conducting questions and answers

[1193] Output: Generated virtual audience, collected responses

[1194] Step 4: Collect and store interview results

[1195] Specific behavior:

[1196] 1. The interviewer agent sends the answers obtained from the virtual subject to the server.

[1197] 2. The server stores the interview results in a database.

[1198] Example: Feedback from young working adults: "The navigation is difficult to understand when using it for the first time."

[1199] Input: Responses from hypothetical subjects

[1200] Data processing: Saving answers

[1201] Output: Archived interview results data

[1202] Step 5: Summarize the data and extract business insights

[1203] Specific behavior:

[1204] 1. The server organizes the interview results stored in the database and sends them to the analyst agent.

[1205] 2. Analyst agents analyze the data and extract specific patterns and business insights.

[1206] Example: "Many users find the navigation difficult to understand on first use."

[1207] Input: Saved interview result data

[1208] Data processing: Data organization and analysis

[1209] Output: Extracted business insights and improvement suggestions

[1210] Step 6: Display insights in a dashboard

[1211] Specific behavior:

[1212] 1. The server displays insights and improvement suggestions obtained from the analyst agent on the user's dashboard.

[1213] 2. Users review insights on the dashboard and consider business improvement measures as needed.

[1214] Example: Consider adding a tutorial guide.

[1215] Input: Insights and improvement suggestions from analyst agents

[1216] Data processing: Displaying insights and improvement suggestions

[1217] Output: Insights and recommendations displayed on the user's dashboard

[1218] By clearly showing the specific operations and inputs / outputs at each processing step, it is possible to explain the entire processing flow of the system in detail, allowing users to efficiently conduct virtual interviews and obtain specific, actionable business insights.

[1219] (Application example 1)

[1220] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1221] In modern businesses, efficiently collecting customer feedback is essential for improving products and services. However, traditional methods have made it difficult to accurately collect and analyze real-time customer feedback in physical stores. Furthermore, there has been a lack of appropriate tools to instantly analyze collected feedback and utilize it as business insights. An effective system is needed to resolve these issues and improve the quality of store operations.

[1222] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1223] In this invention, the server includes means for setting interview topics and hypotheses, means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on that content, means for repeating and additionally conducting interviews, means for analyzing the obtained interview content and extracting business insights and improvement proposals, means for displaying the interview content on smart glasses for collecting feedback from customers in physical stores and recording it as audio, and means for converting the audio data into text and transmitting it to a cloud server. This makes it possible to efficiently collect and analyze real-time customer feedback in physical stores and quickly obtain business insights.

[1224] An "interview question" is the purpose and content of questions set based on the information you want to collect and the hypothesis you want to verify in the virtual interview.

[1225] A "hypothesis" is a premise or condition that is tested by an interview and is confirmed or denied through a series of interviews.

[1226] "Interview content" refers to detailed instructions including specific questions and answer formats that the user will ask in the interview.

[1227] A "user" is someone who uses the interview system to set interview questions and hypotheses, instruct the content of the interview, and analyze the collected data.

[1228] An "interviewer agent" is an agent that conducts an interview based on the interview content set for a virtual subject.

[1229] "Virtual subjects" are hypothetical respondents generated for interviews and based on different personas.

[1230] A "persona" is a fictional character with different attributes and backgrounds that represents a specific target user.

[1231] "Smart glasses" are wearable devices that have the ability to display specific information and record audio data in real time.

[1232] A "cloud server" is a remote server that stores, processes, and analyzes data and is accessed over a network.

[1233] "Voice data" refers to voice response information collected by users or store staff through smart glasses.

[1234] "Convert to text" refers to the process of converting collected voice data into text data.

[1235] "Business insights" are insights and findings that are useful for business improvement and decision-making, based on the analysis of collected data.

[1236] The system includes a means for setting interview topics and hypotheses, a means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on that content, a means for conducting repeated and additional interviews, a means for analyzing the obtained interview content and extracting business insights and improvement proposals, a means for displaying the interview content on smart glasses for collecting feedback from customers in physical stores and recording it as audio, and a means for converting the audio data into text and sending it to a cloud server.

[1237] Specific system implementation includes the following elements:

[1238] 1. Hardware and Software Used:

[1239] Smart glasses: A device used to display the interview content and record audio data.

[1240] Cloud server: A remote server (e.g., AWS) that stores, processes, and analyzes data.

[1241] Speech recognition software: Software that converts voice data into text data (e.g., Google Cloud Speech-to-Text API).

[1242] Large-scale language models: Software used to analyze interview results and extract insights (e.g., OpenAI GPT-4).

[1243] Database: A database (e.g. MySQL) to store the collected data.

[1244] 2. System operation process:

[1245] The server first sets the interview topic and hypotheses. Based on this, the user specifies the interview content, and the interviewer agent conducts the interview with the virtual subjects. The virtual subjects are generated based on personas, and different interviews are repeatedly conducted for each persona.

[1246] In a physical store, the customer wears smart glasses, and the interview content is displayed on the glasses. The customer asks questions to the customer, which are recorded as audio data. This audio data is converted into text in real time and sent to a cloud server.

[1247] The cloud server stores the submitted text data and analyzes it using large-scale language models to extract specific patterns and business insights, which are then displayed on the user's dashboard.

[1248] 3. Example:

[1249] Suppose you want to investigate which aspects of a new store layout interest customers and which aspects dissatisfy them after the new layout is implemented. To do this, set the following hypotheses and questions:

[1250] Hypothesis: "The new layout will make customer flow more smoothly."

[1251] question:

[1252] 1. What did you think of the new layout?

[1253] 2. What area of ​​the store appealed to you the most?

[1254] 3. Were there any aspects that you were dissatisfied with?

[1255] Example prompt sentence:

[1256] Customer Feedback Data:

[1257] 1. What did you think of the new layout?

[1258] "The flow of traffic was good and it was easy to shop." "I got a little lost."

[1259] 2. What area of ​​the store appealed to you the most?

[1260] "Fashion corner" "Food corner"

[1261] 3. Were there any aspects that you were dissatisfied with?

[1262] "The area around the cash register was crowded" "The parking lot was small"

[1263] Based on the customer feedback below, summarize the key themes and insights.

[1264] This system efficiently collects customer feedback in physical stores and instantly analyzes the data, enabling quick and specific business insights.

[1265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1266] Step 1:

[1267] Users log in to the system and set interview questions and hypotheses. These are sent to the cloud server and saved as interview questions. This process clarifies the purpose and direction of the interview.

[1268] Input: Interview objectives, questions, hypotheses

[1269] Output: Saved interview assignments

[1270] Specific operation: Receives user input data, sends it to the cloud server, and stores it in the database.

[1271] Step 2:

[1272] The user provides specific interview questions and answer formats to the cloud server, which then builds a framework for an interviewer agent based on this information, and the interviewer agent then conducts an interview with the virtual subject.

[1273] Input: Interview questions, answers

[1274] Output: Constructed interview framework, collected response data

[1275] Specific operations: Receives user instructions, generates a set of questions for the interviewer agent, and conducts an interview with the virtual subject.

[1276] Step 3:

[1277] The cloud server generates different personas and conducts repeated interviews with virtual subjects based on each persona, thereby collecting feedback from a variety of perspectives.

[1278] Input: Persona information

[1279] Output: A set of response data from each persona

[1280] What it does: Generate multiple personas and gather relevant data by conducting specialized virtual interviews for each.

[1281] Step 4:

[1282] The user wears the smart glasses, and the interview content is displayed on the glasses. Questions are asked to the customer, and their answers are recorded as audio.

[1283] Input: Interview Question

[1284] Output: Recorded audio data

[1285] Specific operation: The interview content is displayed on the smart glasses and the customer's voice responses are recorded.

[1286] Step 5:

[1287] The voice data is converted into text in real time and sent to a cloud server using voice recognition software.

[1288] Input: Audio data

[1289] Output: Text data

[1290] What it does: It uses speech recognition software to convert voice data into text and sends it to a cloud server.

[1291] Step 6:

[1292] The cloud server stores the transmitted text data in a database and analyzes the data using a generative AI model.

[1293] Input: Text data

[1294] Output: Analysis results, business insights

[1295] What it does: Collected text data is stored in a database and patterns and insights are extracted using generative AI models.

[1296] Step 7:

[1297] The server displays the insights and improvement suggestions obtained in real time on the user's dashboard.

[1298] Input: Analysis results

[1299] Output: Business insight dashboard display

[1300] What it does: Updates user dashboards with analysis results, providing real-time business insights.

[1301] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1302] As an embodiment of the present invention, we provide a virtual interview system that utilizes a large-scale language model (LLM) agent and an emotion engine. This system is designed according to the following steps:

[1303] 1. Setting interview questions and hypotheses

[1304] The user logs into the system and inputs the purpose of the interview, the tasks, and the desired hypotheses. The server receives this information and saves it as an interview task. This step clarifies the direction the interview requires and the hypotheses to be verified.

[1305] 2. Instructions and implementation of interview content

[1306] The user specifies specific interview questions and answer formats to the server. For example, the user may set questions to check whether there are any points that are difficult to understand when using the app for the first time. The server builds an interview framework based on these questions and sends it to an interviewer agent. The interviewer agent then conducts the interview with the virtual subject and collects the answers.

[1307] 3. Conducting repeated interviews

[1308] The user sets different personas and repeatedly conducts interviews based on each persona. The server generates a virtual subject for each persona and conducts interviews with them to collect diverse data. For example, the same interview can be conducted with persona A (a young working adult) and persona B (an older housewife) to obtain answers from each perspective.

[1309] 4. Applying the Emotion Engine

[1310] During the interview, the server utilizes an emotion engine to recognize the virtual subject's emotional data in real time. The emotion engine detects the subject's emotional state from their voice tone, choice of words, and reaction speed. This allows the interviewer agent to dynamically adjust the tone and content of questions according to the virtual subject's emotions.

[1311] 5. Summarizing data and extracting business insights

[1312] After all interviews are completed, the server organizes the collected data, and the analyst agent summarizes it. The analyst agent integrates the interview results with sentiment data to extract specific patterns and business insights. The resulting insights and improvement suggestions are displayed on the user's dashboard, allowing them to make real business improvements.

[1313] Specific examples

[1314] Suppose a user wants to evaluate the usability of a new mobile app. The user logs in to the system, sets the interview objective as "evaluating usability," and sets the hypothesis as "the new UI is intuitive to users." Next, the user sets questions to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and asks for free-form responses. Based on this, the server generates an interviewer agent and conducts an interview with the virtual subject.

[1315] The user then sets up two personas: a young working professional and an elderly housewife. The server repeatedly interviews these personas and stores their responses in a database. During the interview, the emotion engine recognizes the virtual subjects' emotions in real time, allowing the interviewer agent to adjust the questions they ask based on the emotional data, thereby gaining deeper insights.

[1316] Once the interviews are complete, the server uses an analyst agent to analyze the data and discover that many users report difficulty with navigation during first-time use. Based on this feedback, including sentiment data, the analyst agent suggests adding a tutorial guide as an improvement and displays it on the user's dashboard.

[1317] In this way, the system efficiently combines virtual interviews and emotion recognition to provide specific, actionable business insights, thereby improving the efficiency and effectiveness of marketing operations.

[1318] The processing flow will be explained below.

[1319] Step 1:

[1320] A user logs in to the system. The user enters their authentication information and accesses the system. The server verifies the user's authentication information, and if authentication is successful, the user is redirected to the user's dashboard.

[1321] Step 2:

[1322] The user inputs the purpose of the interview and the hypothesis from the terminal. The user inputs the purpose of the interview (e.g., "Evaluate the usability of a new mobile app") and the hypothesis (e.g., "The new UI is intuitive to users"). The server receives this information and saves it as an interview assignment.

[1323] Step 3:

[1324] The user sets the interview questions and answer format from the terminal. The user inputs specific questions (e.g., "Which features were difficult to use?") and answer formats (e.g., free text, multiple choice). The server receives these questions and builds the framework for the interview.

[1325] Step 4:

[1326] The server generates an interviewer agent to conduct an interview with the virtual subject. The interviewer agent asks questions to the virtual subject based on the questions received from the server and collects answers. The server records these answers in real time.

[1327] Step 5:

[1328] The user sets different personas from the terminal. The user sets the target persona (e.g., young working people, elderly housewives). The server generates virtual targets based on each persona.

[1329] Step 6:

[1330] The server conducts interviews for each persona. The server generates an interviewer agent for each persona and repeatedly conducts interviews with the virtual subject. The answers from this virtual subject are stored in a database.

[1331] Step 7:

[1332] The server uses an emotion engine to recognize the virtual subject's emotions in real time. The emotion engine detects the subject's emotional state from their tone of voice, choice of words, and reaction speed. During the interview, the emotional data is fed back to the interviewer agent, who dynamically adjusts the tone and content of the questions.

[1333] Step 8:

[1334] The server summarizes the interview data. After ensuring all interviews are complete, the server sends the collected data to the analyst agent, who then summarizes it.

[1335] Step 9:

[1336] Analyst agents extract business insights. Analyst agents analyze summary data and extract useful insights and improvement suggestions (e.g., discovering that "navigation is difficult to understand on first use"). This process also integrates sentiment data for more accurate analysis.

[1337] Step 10:

[1338] The server displays the insights and improvement suggestions on the user's dashboard. The extracted insights and improvement suggestions are visually displayed on the dashboard for the user to use, allowing the user to take concrete improvement measures based on the interview results and sentiment data.

[1339] This series of steps allows users to conduct interview surveys efficiently and effectively, and improve their business based on the obtained data and emotion recognition information.

[1340] Example 2

[1341] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1342] Conventional interview systems are unable to capture the subject's emotions in real time, resulting in less nuanced and reliable answers. They also struggle to collect data from diverse perspectives based on different personas, leading to insufficient extraction of business insights. Furthermore, it is difficult to efficiently analyze the collected data and provide users with immediate feedback and improvement suggestions based on that data.

[1343] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1344] In this invention, the server includes: means for a user to set interview topics and hypotheses; means for a user to specify interview content and for an interviewer agent to conduct interviews with virtual subjects based on the content; means for conducting repeated and additional interviews; means for utilizing an emotion engine to recognize emotional data of the virtual subjects in real time during the interview and dynamically adjust the content and tone of questions; means for generating multiple different personas and conducting interviews with these personas to collect data; means for recording interview results and emotional data in real time and displaying them on a dashboard on a user's operating terminal; and means for analyzing the obtained interview content and emotional data and extracting business insights and improvement proposals. This makes it possible to recognize the emotions of virtual subjects in real time, collect data from various personas, and provide efficient feedback and business insights.

[1345] The "interview topic" refers to the purpose and theme of the interview, and the hypothesis you want to verify.

[1346] A "virtual subject" is a simulated interviewee generated by a computer system.

[1347] An "interviewer agent" is a program or system that conducts interviews with virtual subjects, posing questions and collecting responses.

[1348] The "emotion engine" is a technology that detects emotional states from voice tone, selected words, reaction speed, etc., and recognizes emotional data in real time.

[1349] A "persona" is a fictional character with specific attributes and behavioral patterns, and is used as a representative model for marketing and product design.

[1350] "Business insights" are insights and suggestions that are useful for specific business improvements and decision-making, obtained by analyzing collected data.

[1351] A "dashboard" is an interface that visually displays collected data and analysis results, allowing users to easily view and operate them.

[1352] A "generative AI model" refers to an artificial intelligence model generated by a machine learning algorithm using a large dataset.

[1353] A "prompt" is an instruction used to ask a specific question or give input to an AI model.

[1354] "Response format" refers to the format in which responses to interview questions are collected, and includes free-form and multiple-choice formats.

[1355] This invention is a virtual interview system that utilizes a large-scale language model (LLM) agent and an emotion engine, which allows users to obtain emotion data of virtual subjects in real time and efficiently conduct interviews based on various personas.

[1356] System Configuration

[1357] The system consists of three elements: a server, a terminal, and a user, and operates as follows.

[1358] 1. User Roles

[1359] The user logs in to the system using a terminal and sets the purpose, task, and hypothesis of the interview, which instructs the interviewer agent on the interview content and sends the interview task to the server.

[1360] 2. Server Roles

[1361] The server receives the interview objectives and hypotheses sent by the user and stores them in a database as interview tasks. Next, it generates an interviewer agent and prepares to conduct an interview with the virtual subject. To do this, it uses a generative AI model. The server utilizes an emotion engine to recognize the virtual subject's emotional data in real time and dynamically adjust the content and tone of the questions. The obtained interview content and emotional data are analyzed by the server and provided to the user as business insights and improvement suggestions.

[1362] 3. Roles of the Interviewer Agent and Emotion Engine

[1363] The interviewer agent conducts interviews with virtual subjects and collects their responses. It also analyzes the virtual subjects' emotional data in real time through an emotion engine, dynamically adjusting the content and tone of the interview. The emotion engine detects emotions from voice tone, choice of words, and reaction speed.

[1364] Specific steps and operations

[1365] 1. User login and interview preparation

[1366] The user logs in to the system from their device and sets the purpose of the interview as "evaluating usability" and the hypothesis as "the new UI is intuitive to users." Next, questions are set to confirm whether there are any aspects of the app that are difficult to understand when used for the first time, and responses are asked in the form of free text.

[1367] 2. Create personas and conduct interviews

[1368] The user sets two personas: a young working person and an elderly housewife. The server generates virtual subjects based on these personas, operates an interviewer agent, and repeatedly conducts the same interview with each of them. During this process, the emotion engine recognizes the virtual subjects' emotional data in real time, and the interviewer agent adjusts the questions accordingly.

[1369] 3. Analyzing data and providing business insights

[1370] After the interview is completed, the server sends the collected data to an analyst agent for analysis. The analyst agent integrates the interview results with sentiment data to extract specific patterns and insights. For example, the analyst agent may discover that many users report difficulty with navigation during first-time use and suggest adding a tutorial guide. These insights and improvement suggestions are displayed on the user's dashboard.

[1371] Examples of concrete examples and prompts

[1372] As a concrete example, the following prompt sentence can be used:

[1373] Example prompt sentence:

[1374] "We would like to conduct interviews to evaluate the usability of a new mobile app. The purpose of the interview is to 'evaluate usability', and the hypothesis is that 'the new UI is intuitive to users.' We will set questions to identify any aspects of how to use the app that are difficult to understand upon first use, and will ask for free-form responses. The target personas will be young working people and older housewives."

[1375] With these specific operating procedures and system configurations, the present invention can realize efficient and accurate virtual interviews and the extraction of various business insights.

[1376] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1377] Step 1:

[1378] The user logs into the system from a terminal. The user's ID and password are required as input. The login information is sent to the server, which then references the database for authentication. If this authentication is successful, the user is taken to a screen where they can set the interview topic and hypothesis. Specifically, the user enters their ID and password into the login screen and clicks the "Login" button.

[1379] Step 2:

[1380] The user inputs the purpose, task, and hypothesis of the interview and sends it to the server. The input requires the purpose, task, and hypothesis of the interview. The server receives this information and stores it in the database as an interview task. Specifically, the user inputs the purpose and hypothesis of the interview into the form and presses the "Submit" button.

[1381] Step 3:

[1382] The user specifies the interview content and answer format. The question content and answer format are required as input. The server generates an interview framework based on this and builds an interviewer agent. Specifically, the user inputs the question content and answer format and presses the "Send" button. The server generates a question template using a generative AI model.

[1383] Step 4:

[1384] The server launches an interviewer agent and conducts an interview with the virtual subject. The inputs required are an interview framework and a profile of the virtual subject. The server instructs the interviewer agent on the content of the interview, and the agent asks questions to the virtual subject. Specifically, the server launches the agent and has it begin asking questions to the virtual subject.

[1385] Step 5:

[1386] The server utilizes an emotion engine to recognize the virtual subject's emotional data in real time. Inputs include the virtual subject's tone of voice, selected words, and reaction speed. The server analyzes this data with the emotion engine to detect their emotional state. Specifically, the server integrates the emotion engine into the interviewer agent and collects emotional data during the interview.

[1387] Step 6:

[1388] The interviewer agent dynamically adjusts the content and tone of questions based on the emotional data of the virtual subject. The emotional data and the current question are required as input. The server modifies the question based on the emotional data and sends instructions to the interviewer agent. Specifically, the server dynamically changes the next question using the results of the emotional data analysis.

[1389] Step 7:

[1390] The user sets up different personas. The persona's attributes (age, occupation, gender, etc.) are required as input. The server generates multiple virtual subjects based on this and conducts repeated interviews with each of them. Specifically, the user enters the persona information and presses the "Submit" button.

[1391] Step 8:

[1392] The server analyzes the collected interview content and sentiment data. The collected interview data and sentiment data are required as input. The server uses analyst agents to analyze the data and generate business insights and improvement proposals. Specifically, the server sends the data to the analyst agents and displays the analysis results on a dashboard.

[1393] Step 9:

[1394] Users can view analysis results on the dashboard and check improvement proposals. Analysis result data is required as input. Business insights and improvement proposals are displayed as output. In concrete terms, users access the dashboard and check the analysis results.

[1395] (Application example 2)

[1396] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1397] Conventional methods for evaluating advertising campaigns have made it difficult to quickly and efficiently collect target audience emotional responses and specific feedback. Furthermore, evaluating the effectiveness of advertising campaigns individually for different personas requires a significant amount of time and effort. The present invention aims to solve these problems by efficiently evaluating advertising campaigns and quickly obtaining specific insights that take into account the target audience's emotional responses.

[1398] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1399] In this invention, the server includes means for setting interview topics and hypotheses, means for a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content, means for setting a plurality of different personas and repeatedly conducting interviews based on each persona, means for using an emotion engine to recognize emotional data of the virtual subject in real time during the interview and for the interviewer agent to dynamically adjust the tone and content of questions based on the emotion, and means for analyzing the obtained interview content and extracting business insights and improvement proposals. This makes it possible to efficiently evaluate the effectiveness of an advertising campaign while taking into account the emotional responses of the target audience and quickly obtain specific feedback and business insights.

[1400] "Means for setting interview questions and hypotheses" is a function that allows system users to specifically input the objectives and goals of the advertising campaign they wish to evaluate.

[1401] "A means for the user to specify the interview content and for the interviewer agent to conduct a virtual interview based on that" is a function that conducts an interview with a virtual subject based on the questions specified by the user.

[1402] "A means of setting up multiple different personas and repeatedly conducting interviews based on each persona" is a function that virtually sets up different target groups and conducts interviews with each of them to collect data.

[1403] "Using an emotion engine to recognize the emotional data of virtual subjects in real time during an interview, allowing the interviewer agent to dynamically adjust the tone and content of questions based on their emotions" is a function that grasps the emotional state of the subject during the interview and flexibly changes the interview approach accordingly.

[1404] "Means of analyzing the interview content obtained and extracting business insights and improvement proposals" is a function that analyzes the collected interview data and derives specific insights and improvement proposals regarding the effectiveness of advertising campaigns.

[1405] The present invention relates to a system for evaluating the effectiveness of an advertising campaign, which operates in the following manner.

[1406] First, users log in to the system and set the objectives and hypotheses for their advertising campaign. For example, they can set a hypothesis that "the new advertisement will be very popular with young people." This setting information is then stored on the server.

[1407] Next, the user specifies the specific content of the interview. Questions can be set such as "What was your first impression when you saw this advertisement?" or "What emotions did you feel when you saw this advertisement?" and responses can be requested in the form of free description or multiple choice. Based on this, the server generates an interviewer agent and begins interviewing the virtual subject.

[1408] In this process, multiple personas are created to represent different target demographics (for example, "male high school students" and "female university students"), and interviews are conducted repeatedly. This process makes it possible to collect a wide variety of data.

[1409] During the interview, the server uses an emotion engine to recognize the virtual subject's emotional data in real time. The emotion engine detects the subject's emotional state from their voice tone, choice of words, reaction speed, etc. Based on the recognized emotional data, the interviewer agent can dynamically adjust the tone and content of their questions to extract deeper insights.

[1410] Once the interviews are complete, the server organizes the collected data, which is then summarized by an analysis agent. The analysis agent then integrates the interview results with sentiment data to extract specific patterns and business insights. The resulting insights and recommendations are displayed on the user's dashboard, allowing them to use them to improve their actual advertising campaigns.

[1411] This invention allows for efficient evaluation of advertising campaigns that take into account the emotional responses of the target audience. For example, it is possible to quickly obtain specific feedback such as how high school boys felt when they saw the advertisement, or how college girls felt when they saw the advertisement.

[1412] Example prompt sentence:

[1413] "How do high school boys feel when they see this ad?"

[1414] "What emotions did college girls have when they saw this ad?"

[1415] Thus, the present invention is a system that utilizes a large-scale language model and an emotion engine to enable the effectiveness of advertising campaigns to be obtained from virtual audiences that are close to the actual target demographic, thereby maximizing the effectiveness of advertising and providing important insights for formulating more effective advertising strategies for the target demographic.

[1416] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1417] Step 1:

[1418] Users log in to the system from their devices and set the objectives and hypotheses for their advertising campaign. Hypotheses such as "the new advertisement will be very popular with young people" are entered as input, and the server receives and stores this information. Based on this configuration information, the server processes the data to determine the direction of the interviews.

[1419] Step 2:

[1420] The user inputs specific interview content from the terminal. For example, they input questions such as, "What was your first impression when you saw this advertisement?" or "What emotions did you feel when you saw this advertisement?" The server receives these and performs data calculations to pass instructions to the interviewer agent. As a result, a question framework for the interviewer agent is constructed.

[1421] Step 3:

[1422] Users can set different target demographics (personas) on their devices. For example, they can input personas such as "high school boys" or "university girls." The server receives this data and processes it to generate multiple virtual targets based on the specified personas.

[1423] Step 4:

[1424] The interviewer agent conducts interviews with virtual subjects based on instructions from the server. When questions are presented to the subjects, their responses are collected. The input is the question and the virtual subject's prompt, and the output is the collected response data.

[1425] Step 5:

[1426] While the virtual subject is answering, the server uses an emotion engine to recognize and analyze emotional data from the answers in real time. The input is the collected answer data and data on the virtual subject's reactions, and the output is the detected emotional data. Based on this emotional data, the interviewer agent dynamically adjusts the tone and content of the questions.

[1427] Step 6:

[1428] Once the interview is over, the server organizes the collected interview data and sentiment data, and the data is summarized by the analysis agent. The input is the collected response data and sentiment data, and the output is specific patterns and business insights.

[1429] Step 7:

[1430] The server displays the analysis results on the user's dashboard, allowing the user to gain specific insights into the effectiveness of their advertising campaigns and areas for improvement. The input is the analyzed data and business insights, and the output is the resulting information displayed on the dashboard.

[1431] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1432] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1433] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1434] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1435] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1436] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1437] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1438] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1439] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1440] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1441] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1442] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1443] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1444] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1445] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1446] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1447] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1448] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1449] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1450] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1451] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1452] The following is further disclosed regarding the above embodiment.

[1453] (Claim 1)

[1454] A means of setting interview questions and hypotheses;

[1455] A means for allowing a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content;

[1456] means for conducting repeat and additional interviews;

[1457] A means to analyze the interview content obtained and extract business insights and improvement proposals,

[1458] A system including:

[1459] (Claim 2)

[1460] 10. The system of claim 1, further comprising means for generating a plurality of different personas as virtual subjects and conducting interviews with them.

[1461] (Claim 3)

[1462] 10. The system of claim 1, further comprising means for recording results of the interview in real time and displaying them on a user dashboard.

[1463] "Example 1"

[1464] (Claim 1)

[1465] a means for users to log in and set interview questions and hypotheses;

[1466] A means of inputting the purpose, tasks, and expected hypotheses of the interview;

[1467] A means for allowing a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content;

[1468] A method for setting different personas and repeatedly conducting interviews based on each persona,

[1469] a means for an interviewer agent to conduct interviews with virtual subjects and collect responses;

[1470] A means of storing the collected interview content as data,

[1471] Analyst Agents organize and analyze the collected data to extract business insights and improvement proposals.

[1472] A means to display extracted insights and improvement suggestions on the user's dashboard,

[1473] A system including:

[1474] (Claim 2)

[1475] 10. The system of claim 1, further comprising means for establishing different personas, generating virtual subjects for them, and conducting interviews.

[1476] (Claim 3)

[1477] 10. The system of claim 1, further comprising means for recording the results of the interviews in a database and displaying the analysis results on a user dashboard.

[1478] "Application Example 1"

[1479] (Claim 1)

[1480] A means of setting interview questions and hypotheses;

[1481] A means for allowing a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content;

[1482] means for conducting repeat and additional interviews;

[1483] A means to analyze the interview content obtained and extract business insights and improvement proposals,

[1484] A means of displaying and recording interview content on smart glasses for collecting feedback from customers in physical stores;

[1485] means for converting the voice data into text and transmitting the text to a cloud server;

[1486] A system including:

[1487] (Claim 2)

[1488] 10. The system of claim 1, further comprising means for generating a plurality of different personas as virtual subjects and conducting interviews with them.

[1489] (Claim 3)

[1490] 10. The system of claim 1, further comprising means for recording results of the interview in real time and displaying them on a user's dashboard.

[1491] "Example 2: Combining Emotion Engines"

[1492] (Claim 1)

[1493] a means for users to define interview questions and hypotheses;

[1494] a means for allowing a user to specify the content of the interview and for an interviewer agent to conduct an interview with the virtual subject based on the specified content;

[1495] means for conducting repeat and additional interviews;

[1496] A means of utilizing an emotion engine to recognize the emotional data of virtual subjects in real time during the interview and dynamically adjust the content and tone of questions;

[1497] The interview content and sentiment data obtained are analyzed to extract business insights and improvement proposals.

[1498] A system including:

[1499] (Claim 2)

[1500] 10. The system of claim 1, further comprising means for generating a plurality of different personas and conducting interviews therewith to collect data.

[1501] (Claim 3)

[1502] 10. The system of claim 1, further comprising means for recording interview results and emotion data in real time and displaying them on a dashboard on a user's operating terminal.

[1503] "Application example 2 when combining emotion engines"

[1504] (Claim 1)

[1505] A means of setting interview questions and hypotheses;

[1506] A means for allowing a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content;

[1507] A method of setting up multiple different personas and repeatedly conducting interviews based on each persona,

[1508] A means for using an emotion engine to recognize the virtual subject's emotional data in real time during the interview and for the interviewer agent to dynamically adjust the tone and content of questions based on the emotion;

[1509] A means to analyze the interview content obtained and extract business insights and improvement proposals,

[1510] A system including:

[1511] (Claim 2)

[1512] 10. The system of claim 1, further comprising means for generating a plurality of different personas as virtual subjects and evaluating the advertising campaign against them.

[1513] (Claim 3)

[1514] 10. The system of claim 1, further comprising means for recording interview results and sentiment data in real time and displaying them on a user's dashboard. [Explanation of symbols]

[1515] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of setting interview questions and hypotheses; A means for allowing a user to specify interview content and for an interviewer agent to conduct a virtual interview based on the specified content; means for conducting repeat and additional interviews; A means to analyze the interview content obtained and extract business insights and improvement proposals, A system including:

2. The system of claim 1 , further comprising means for generating a plurality of different personas as virtual subjects and conducting interviews with them.

3. 10. The system of claim 1, further comprising means for recording the results of the interview in real time and displaying them on a user dashboard.

Images