system

Abstract

Provide a system. 【Solution means】Scenario selection means, Customer service information acquisition means, Scenario generation means, User input reception means, Customer service response generation means, Conversation log storage means, Feedback generation means, Evaluation display means, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a demand for providing a practical practice environment for effectively improving the closing proposal skills of new employees. However, conventional training methods have problems that it is difficult to reproduce interactions with actual customers and that practice using real scenarios and responses cannot be carried out. Furthermore, feedback is often delayed, and non-immediate evaluation hinders effective skill improvement.

Means for Solving the Problems

[0005] To solve this problem, the present invention provides a system comprising multiple means. This system includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, and an evaluation display means. This allows the user to freely select a scenario and conduct a virtual conversation using the actual customer role's responses. The conversation log is saved and immediately analyzed, so quick and appropriate feedback is provided. In this way, the effective improvement of new employees' closing proposal skills is achieved.

[0006] The "scenario selection method" is a function that allows users to select a virtual scenario for practice.

[0007] The "customer role information acquisition method" is a function that retrieves information about fictional characters acting as customers from a database.

[0008] The "scenario generation method" is a function that generates practice scenarios based on acquired customer role information.

[0009] "User input receiving means" refers to a function that receives input or responses from the user.

[0010] The "customer role response generation means" is a function that generates a virtual response from a customer based on user input.

[0011] The "conversation log saving method" is a function that saves logs of virtual conversations between the user and the customer.

[0012] The "feedback generation method" is a function that analyzes saved conversation logs and generates feedback for the user.

[0013] "Evaluation display means" refers to a function that displays the generated feedback and evaluations to the user. [Brief explanation of the drawing]

[0014] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0015] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The system of this invention is designed to provide a practice environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users (new employees), and its specific operation is described below.

[0036] server

[0037] The server plays a central role in generating scenarios and customer responses.

[0038] 1. Scenario selection method:

[0039] The server receives the scenario selection information specified by the user.

[0040] Specific example: The user selects the scenario "consultation on purchasing home appliances."

[0041] 2. Means of obtaining customer role information:

[0042] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[0043] Specific example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[0044] 3. Scenario generation method:

[0045] Based on the acquired customer information, a scenario is generated from a pre-prepared template.

[0046] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[0047] 4. User input receiving means:

[0048] The server receives user input from the terminal.

[0049] Specific example: Receive information entered by the user, such as "What kind of product are you looking for?".

[0050] 5. Means for generating customer role responses:

[0051] The server analyzes the received user input and generates the next response from the customer based on that input.

[0052] Specific example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[0053] 6. Means of saving conversation logs:

[0054] The server saves logs of virtual conversations between users and customers.

[0055] Specific example: Save each exchange in a conversation in chronological order.

[0056] 7. Feedback generation means:

[0057] Based on the conversation log, it generates an evaluation and feedback on the user's closing suggestion.

[0058] Specific example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[0059] 8. Means of displaying evaluations:

[0060] The server sends the generated feedback to the terminal and displays it to the user.

[0061] Specific example: Display the feedback content on the user's device.

[0062] terminal

[0063] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[0064] 1. Scenario selection screen:

[0065] The user is prompted to select a scenario on the initial screen.

[0066] Specific example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[0067] 2. User input screen:

[0068] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[0069] Specific example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[0070] 3. Response display screen:

[0071] Displays the customer's response to the user's input.

[0072] Specific example: The server displays the response, "I am considering purchasing a refrigerator. Please tell me the differences between products from company A and company B."

[0073] 4. Feedback screen:

[0074] After the conversation ends, the feedback sent from the server will be displayed.

[0075] Specific example: Display a review stating, "The closing proposal was very clear, but it lacked a specific price."

[0076] User

[0077] Users operate this system to simulate interactions with actual customers.

[0078] 1. Scenario Selection:

[0079] The user selects the scenario they want to practice from the scenario selection screen on their device.

[0080] Example: Select "Consultation on purchasing home appliances".

[0081] 2. The progress of the virtual conversation:

[0082] The user progresses the conversation by responding to messages from a customer displayed on their device.

[0083] Example: Continue the conversation with the customer by typing, "What kind of product are you looking for?"

[0084] 3. Closing proposal:

[0085] Based on the flow of the conversation, the user makes a closing proposal.

[0086] Specific example: "Our refrigerators from Company A use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[0087] 4. Feedback received:

[0088] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[0089] Specific example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[0090] In this way, by using this system, new employees can efficiently improve their closing proposal skills in a practical environment.

[0091] The following describes the processing flow.

[0092] Step 1:

[0093] The user opens the scenario selection screen using their device and specifies the scenario they want to practice. The scenario selection information is sent to the server.

[0094] Step 2:

[0095] The server analyzes the scenario selection information received from the user and retrieves customer role information related to the selected scenario from the database.

[0096] Step 3:

[0097] Based on the customer information acquired by the server, a scenario is generated from a pre-prepared template. The generated scenario data is then sent to the terminal.

[0098] Step 4:

[0099] Based on the scenario data received from the server, the terminal displays an initial message to the user, who is playing the role of a customer. The user then enters a response to the initial message.

[0100] Step 5:

[0101] User input is sent from the terminal to the server. The server analyzes the user input and generates the next customer response based on its content.

[0102] Step 6:

[0103] The server sends the next response generated by the customer role to the terminal. The terminal displays this to the user. The user then enters their response again.

[0104] Step 7:

[0105] Steps 5 and 6 are repeated, and a virtual conversation between the user and the customer progresses. The user makes closing suggestions based on the flow of the conversation and continues to input.

[0106] Step 8:

[0107] Once the conversation ends, the terminal sends all conversation logs to the server. The server saves the conversation logs and begins analysis.

[0108] Step 9:

[0109] The server generates an evaluation and feedback on the user's closing suggestion based on the conversation log. The generated feedback is then sent to the terminal.

[0110] Step 10:

[0111] The device displays feedback received from the server to the user. The user reviews the feedback and uses it to improve their next practice session.

[0112] (Example 1)

[0113] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0114] Providing an effective training environment for new employees to practically improve their closing proposal skills is challenging. Traditional training lacks real-time feedback and practice using concrete scenarios, hindering the development of skills relevant to actual work. To address this challenge, a system is needed that provides real-time quantitative and qualitative feedback through virtual customer interactions.

[0115] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0116] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a user input means via a terminal, a scenario display means via a terminal, a customer role response display means via a terminal, a feedback display means via a terminal, a means for acquiring customer role information from a database via the server, a scenario generation means via the server, a user input analysis means via the server, a customer role message display means via a terminal, and a means for generating and displaying evaluations and feedback based on the conversation log. This enables the user to effectively practice dialogue skills and closing proposal skills through real-time virtual conversations based on scenarios and to receive detailed feedback.

[0117] A "scenario selection method" is a function that provides an interface for users to select the scenario they want to practice.

[0118] The "customer role information acquisition method" is a function that retrieves information about customer roles related to a scenario from a database.

[0119] The "scenario generation method" is a function that generates a scenario using a template based on the acquired customer role information.

[0120] A "user input receiving means" is a function that receives input from the user and analyzes it.

[0121] The "customer role response generation means" is a function that generates the next response of the customer role based on user input.

[0122] The "conversation log saving method" is a function that saves a chronological log of a virtual conversation between the user and the customer.

[0123] The "feedback generation method" is a function that generates evaluations and feedback on the user's closing suggestion based on the saved conversation log.

[0124] "Evaluation display means" refers to a function that displays the generated feedback to the user.

[0125] "User input means via terminal" refers to a function that accepts user input in real time.

[0126] "Scenario display means via terminal" refers to a function that displays the generated scenario to the user.

[0127] "Means for displaying customer responses via a terminal" refers to a function that displays the customer's response to user input.

[0128] "Means of displaying feedback via the terminal" refers to a function that displays the generated feedback to the user after the conversation has ended.

[0129] "Means of retrieving customer role information from the database by the server" refers to the function of retrieving customer role information from the database.

[0130] "Server-based scenario generation method" refers to a function that generates scenarios using templates based on customer information.

[0131] "Server-based user input analysis means" refers to a function that analyzes user input and generates the next customer response based on that analysis.

[0132] "Means for displaying customer-role messages via a terminal" refers to a function that displays messages from the customer role to the user.

[0133] "Means for generating and displaying evaluations and feedback based on conversation logs" refers to a function that generates evaluations and feedback on a user's closing suggestion based on saved conversation logs and displays them to the user.

[0134] The present invention provides a training environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users, and each component works in cooperation with the others.

[0135] System Configuration

[0136] server

[0137] The server plays a central role in the system, providing the means to generate scenarios and customer responses. The server has the following functions:

[0138] 1. Scenario Selection Method

[0139] The server receives scenario selection information specified by the user.

[0140] Example: The user selects the scenario "Consultation on purchasing home appliances".

[0141] 2. Means for obtaining customer role information

[0142] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[0143] Example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[0144] 3. Scenario generation means

[0145] The server generates a scenario from a template based on the customer role information it has acquired.

[0146] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[0147] 4. User input receiving means

[0148] The server receives user input from the terminal.

[0149] Example: Receive information entered by the user, such as "What kind of product are you looking for?".

[0150] 5. Customer response generation means

[0151] The server analyzes the user's input and generates the next customer's response based on that input.

[0152] Example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[0153] 6. Means of saving conversation logs

[0154] The server saves logs of virtual conversations between users and customers.

[0155] Example: Save all conversations in chronological order.

[0156] 7. Feedback generation means

[0157] The server generates an evaluation and feedback on the user's closing suggestion based on the saved conversation logs.

[0158] Example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[0159] 8. Evaluation display means

[0160] The server sends the generated feedback to the terminal and displays it to the user.

[0161] Example: Display the feedback content on the user's device.

[0162] terminal

[0163] The terminal provides an interface for users to engage in virtual conversations through communication with the server. The terminal's functions are as follows:

[0164] 1. Scenario Selection Screen

[0165] The user is prompted to select a scenario on the initial screen.

[0166] Example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[0167] 2. User input screen

[0168] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[0169] Example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[0170] 3. Response display screen

[0171] Displays the customer's response to the user's input.

[0172] Example: The server will display the response, "I am considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[0173] 4. Feedback screen

[0174] After the conversation ends, the feedback sent from the server will be displayed.

[0175] Example: Display a review such as, "It was a very clear closing proposal, but it lacked a specific price."

[0176] User

[0177] The user operates this system to simulate interactions with actual customers. Follow these steps:

[0178] 1. Scenario Selection

[0179] The user selects the scenario they want to practice from the scenario selection screen on their device.

[0180] Example: Select "Consultation on purchasing home appliances".

[0181] 2. Progress of the virtual conversation

[0182] The user progresses the conversation by responding to messages from a customer displayed on their device.

[0183] Example: Continue the conversation with the customer by typing "What kind of product are you looking for?"

[0184] 3. Closed proposal

[0185] Based on the flow of the conversation, the user makes a closing proposal.

[0186] Example: "Our A Company refrigerators use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[0187] 4. Receiving Feedback

[0188] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[0189] Example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[0190] Example of a prompt

[0191] "Analyze the following user input and generate an appropriate response to the virtual customer's question: 'What products are you looking for?'"

[0192] In this way, by using this system, new employees can efficiently improve their closing proposal skills in a practical environment.

[0193] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0194] Step 1: Scenario Selection

[0195] The server receives scenario selection information sent from the terminal. Based on this input information, the server recognizes the selected scenario and prepares for the next processing. For example, if a user selects the scenario "consultation on purchasing home appliances," that information is sent to the server.

[0196] Step 2: Obtain customer role information

[0197] The server uses the scenario selection information to retrieve relevant customer role information from the database. This process involves executing database queries and extracting the necessary data. The retrieved data includes customer attribute information and past sales history. For example, the server retrieves attribute information and past sales history for a "customer considering purchasing home appliances."

[0198] Step 3: Scenario Generation

[0199] The server generates a scenario using a template based on the customer role information it has acquired. In this process, data is applied to the template to generate a specific conversation scenario. The generated scenario includes an initial message and questions. For example, the server generates a scenario based on "consultation about purchasing home appliances" and creates an initial message such as "Hello, I'm thinking about purchasing home appliances."

[0200] Step 4: Display the scenario

[0201] The terminal displays the scenario sent from the server to the user. In this process, the terminal converts the received data into an appropriate layout for display on the screen. It receives scenario data from the server as input and displays it to the user as output. For example, the terminal displays the message "Hello, I'm thinking about buying home appliances" to the user.

[0202] Step 5: User response input

[0203] The user enters a response to a customer-facing message displayed on the terminal. The entered data is sent from the terminal to the server. For example, the user might enter "What product are you looking for?"

[0204] Step 6: Receiving and analyzing user input

[0205] The server analyzes user input received from the terminal. This process uses natural language processing techniques to understand the user's intent and generate the next customer response. It receives the user's message as input, analyzes it, and then generates the next response data. For example, the server analyzes the input "What kind of product are you looking for?" and generates the response "I'm considering a refrigerator. Could you tell me the difference between products from company A and company B?"

[0206] Step 7: Customer's reaction

[0207] The terminal displays customer responses generated by the server to the user. In this process, it receives response data from the server and displays it appropriately on the screen. It receives response data as input and displays it to the user as output. For example, the terminal displays, "I'm considering buying a refrigerator. Could you tell me the differences between products from company A and company B?"

[0208] Step 8: Saving the conversation log

[0209] The server saves logs of virtual conversations between the user and the customer. This process stores all conversational exchanges chronologically in a database. It takes conversational data as input and saves it to the database as output. For example, the server saves each individual exchange in a conversation as a log.

[0210] Step 9: Generating Feedback

[0211] The server generates an evaluation and feedback on the user's closing proposal based on the saved conversation logs. This process analyzes the conversation logs and creates feedback based on evaluation criteria. It takes conversation logs as input and generates feedback data as output. For example, the server might generate feedback such as, "This was a very clear closing proposal, but it lacked a specific price offer."

[0212] Step 10: Displaying Feedback

[0213] The terminal displays feedback sent from the server to the user. This process converts the feedback data into an appropriate layout for display on the screen. It receives feedback data as input and displays it to the user as output. For example, the terminal displays the feedback, "It was a very clear closing offer, but it lacked specific pricing information."

[0214] (Application Example 1)

[0215] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0216] For new employees to efficiently improve skills such as presentations and content proposals, practice in a realistic environment is necessary. However, many companies have limited employees and resources for such practice, making it difficult to provide effective training. Furthermore, there are challenges such as limited opportunities for individual feedback and difficulty in identifying areas for improvement through self-assessment.

[0217] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0218] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a response training means including scenario selection information, and a prompt sentence generation means. This allows the user to practice in real time through interaction with a virtual customer and receive individualized feedback.

[0219] "Scenario selection method" refers to the means by which users select a scenario for practice.

[0220] "Method for obtaining customer role information" refers to a means for obtaining information about customer roles related to the selected scenario from a database.

[0221] A "scenario generation method" is a means of generating a scenario from a pre-prepared template based on acquired customer role information.

[0222] "User input receiving means" refers to means for receiving user input.

[0223] "Customer role response generation means" refers to means for generating the next response of the customer role based on the received user input.

[0224] A "conversation log saving method" is a means for saving logs of virtual conversations between a user and a customer.

[0225] A "feedback generation method" is a means of generating evaluations and feedback on a user's closing suggestion based on conversation logs.

[0226] "Evaluation display means" refers to a means for displaying the generated feedback to the user.

[0227] A "response training method including scenario selection information" is a means for training responses based on scenario selection information specified by the user.

[0228] A "prompt message generation means" is a means for generating prompt messages that users will use.

[0229] The present invention is a system for efficiently improving the presentation skills and content proposal abilities of new employees. This system includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a response training means including scenario selection information, and a prompt sentence generation means.

[0230] server

[0231] Scenario selection method

[0232] The server receives information from the user about the scenario they want to practice. For example, if the user selects "New Product Presentation Scenario," the server processes this information.

[0233] Customer information acquisition method

[0234] The server retrieves customer role information from the database based on the selected scenario. This information includes customer profiles and past conversation history.

[0235] Scenario generation method

[0236] Based on the customer information obtained, the server generates an appropriate scenario from pre-prepared templates. For example, it might generate a "presentation scenario emphasizing the advantages of a new product."

[0237] User input receiving means

[0238] The server receives user input (for example, a question like, "How is the new product better?") in real time.

[0239] Customer response generation means

[0240] The server analyzes user input and generates the next response from the customer role based on that input. For example, in response to user input, it might generate a response such as, "The new product incorporates the latest technology and is very easy to use."

[0241] Conversation log saving method

[0242] The server stores a chronological log of the conversation between the user and the customer. This log is used later to generate feedback.

[0243] Feedback generation means

[0244] Based on the conversation log, the server generates user closing suggestions and an overall evaluation and feedback on the presentation. For example, it might provide feedback such as, "The presentation was generally easy to understand, but it lacked detailed explanations of the specifications."

[0245] Evaluation display means

[0246] The server sends the generated feedback to the user's device, allowing the user to check their performance.

[0247] terminal

[0248] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[0249] Scenario selection screen

[0250] The initial screen allows the user to select a scenario. For example, a screen might be displayed where the user can select "New Product Presentation Scenario."

[0251] User input screen

[0252] It displays an initial message from a customer based on a scenario and accepts user input in real time. For example, it might display the message, "Hello, please give me a presentation on the new product," and then receive a response from the user.

[0253] Response display screen

[0254] It displays the customer's response to the user's input. For example, it might display the server's response: "The new product incorporates the latest technology and is very easy to use."

[0255] Feedback screen

[0256] After the conversation ends, the server displays the feedback it has received. For example, it might show an evaluation such as, "The presentation was generally easy to understand, but it lacked detailed explanations of the specifications."

[0257] User

[0258] Scenario Selection

[0259] The user selects the scenario they want to practice from the scenario selection screen on their device. For example, they might select "New Product Presentation Scenario."

[0260] Progress of the virtual conversation

[0261] The user progresses the conversation by responding to messages from a customer displayed on the device. For example, they might type, "How is the new product better?" to continue the conversation.

[0262] Closed proposal

[0263] Based on the flow of the conversation, the user makes a closing suggestion. For example, they might suggest, "This new product is reasonably priced, so please give it a try."

[0264] Feedback received

[0265] After the conversation ends, review the feedback displayed on the device and use it to improve your next practice session. For example, you might receive feedback such as, "The presentation was generally easy to understand, but there was a lack of detailed explanation of the specifications," and use that to identify areas for improvement.

[0266] Specific examples and prompt statements

[0267] This example illustrates how new content creators can learn to present new content. An example of a prompt is: "Our new content offers clear visuals and intuitive controls that will excite viewers. Is there anything specific you're concerned about?" In this way, newcomers can effectively improve their skills by utilizing scenarios provided by the server and real-time feedback.

[0268] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0269] Step 1:

[0270] The server receives scenario selection information from the user through a scenario selection mechanism. Specifically, the user selects the scenario they want to practice and sends that information to the server. The input is scenario selection information, and the output is the selected scenario information.

[0271] Step 2:

[0272] The server uses a customer role information acquisition mechanism to retrieve customer role information related to the selected scenario from the database. This includes data such as the customer role's profile and past conversation history. The input is the selected scenario information, and the output is the retrieved customer role information.

[0273] Step 3:

[0274] The server uses a scenario generation mechanism to generate a scenario based on the acquired customer role information. A specific scenario is created using a template. The input is customer role information and a template, and the output is the generated scenario.

[0275] Step 4:

[0276] The server receives user input through a user input receiving mechanism. The user inputs questions and suggestions for the customer role based on a scenario. The input is direct information from the user, and the output is that same input information.

[0277] Step 5:

[0278] The server uses a customer role response generation mechanism to analyze the received user input and generates the next customer role response using a generation AI model. The input is the user's input information, and the output is the generated customer role response.

[0279] Step 6:

[0280] The server uses a conversation log storage mechanism to save logs of virtual conversations between the user and the customer. The input consists of the user's input information and the customer's responses, and the output is the saved conversation log.

[0281] Step 7:

[0282] The server uses feedback generation means to generate an evaluation and feedback on the user's closing proposal based on the stored conversation log. The input is the stored conversation log, and the output is the generated feedback information.

[0283] Step 8:

[0284] The server displays the generated feedback to the user through evaluation display means. The feedback content is displayed on the user's terminal. The input is the feedback information, and the output is the feedback displayed on the user's terminal.

[0285] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0286] The system of the present invention is designed to provide a practice environment for effectively improving the closing proposal skills of new employees. This system is composed of a server, a terminal, and a user (new employee), and its specific operations will be described below.

[0287] Server

[0288] The server plays a central role in generating scenarios and generating responses of customer service roles. Also, it analyzes the user's emotion using an emotion engine and reflects it in the system's response

[0289] 1. Scenario selection means:

[0290] The server receives scenario selection information specified by the user.

[0291] Specific example: The user selects a scenario of "consultation on purchasing household appliances".

[0292] 2. Customer service information acquisition means:

[0293] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[0294] Specific example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[0295] 3. Scenario generation method:

[0296] Based on the acquired customer information, a scenario is generated from a pre-prepared template.

[0297] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[0298] 4. User input receiving means:

[0299] The server receives user input from the terminal.

[0300] Specific example: Receive information entered by the user, such as "What kind of product are you looking for?".

[0301] 5. Means for generating customer role responses:

[0302] The server analyzes the received user input and generates the next response from the customer based on that input.

[0303] Specific example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[0304] 6. Means of saving conversation logs:

[0305] The server saves logs of virtual conversations between users and customers.

[0306] Specific example: Save each exchange in a conversation in chronological order.

[0307] 7. Feedback generation means:

[0308] Generate evaluations and feedback on the user's close proposals based on the conversation log.

[0309] Specific example: Generate feedback such as "It was a very clear close proposal, but the specific presentation of the price was lacking."

[0310] 8. Evaluation display means:

[0311] The server sends the generated feedback to the terminal and displays it to the user.

[0312] Specific example: Display the feedback content on the user's terminal.

[0313] 9. Emotion engine:

[0314] The server uses the emotion engine to analyze the user's expressions and vocal tones and recognize emotions.

[0315] Specific example: When the user shows a dissatisfied expression, generate a scenario that analyzes the emotion data and adds positive information.

[0316] Terminal

[0317] The terminal provides an interface for the user to conduct a virtual conversation through communication with the server.

[0318] 1. Scenario selection screen:

[0319] Let the user select a scenario on the initial screen.

[0320] Specific example: Display a screen for selecting the "Household Appliance Purchase Consultation" scenario.

[0321] 2. User input screen:

[0322] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[0323] Specific example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[0324] 3. Response display screen:

[0325] Displays the customer's response to the user's input.

[0326] Specific example: The server displays the response, "I am considering purchasing a refrigerator. Please tell me the differences between products from company A and company B."

[0327] 4. Emotion Recognition Interface:

[0328] The device uses cameras and microphones to analyze the user's facial expressions and voice tone.

[0329] Specific example: Sending the tone of voice and facial expressions of a user to the server in real time when they speak.

[0330] 5. Feedback screen:

[0331] After the conversation ends, the feedback sent from the server will be displayed.

[0332] Specific example: Display a review stating, "The closing proposal was very clear, but it lacked a specific price."

[0333] User

[0334] Users operate this system to simulate interactions with actual customers. Furthermore, an emotion engine analyzes those emotions in real time.

[0335] 1. Scenario Selection:

[0336] The user selects the scenario they want to practice from the scenario selection screen on their device.

[0337] Example: Select "Consultation on purchasing home appliances".

[0338] 2. The progress of the virtual conversation:

[0339] The user progresses the conversation by responding to messages from a customer displayed on their device.

[0340] Example: Continue the conversation with the customer by typing, "What kind of product are you looking for?"

[0341] 3. Closing proposal:

[0342] Based on the flow of the conversation, the user makes a closing proposal.

[0343] Specific example: "Our refrigerators from Company A use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[0344] 4. Emotional monitoring:

[0345] During the conversation, the user's facial expressions and voice tone are analyzed in real time, and that data is sent to the server.

[0346] Specific example: If a user shows signs of anxiety, this is analyzed and influences the next scenario.

[0347] 5. Feedback received:

[0348] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[0349] Specific example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[0350] Thus, by using this system, new employees can not only efficiently improve their closing proposal skills in a practical environment, but also engage in more advanced practice that takes user emotions into consideration.

[0351] The following describes the processing flow.

[0352] Step 1:

[0353] The user opens the scenario selection screen using their device and specifies the scenario they want to practice. The scenario selection information is sent to the server.

[0354] Step 2:

[0355] The server analyzes the scenario selection information received from the user and retrieves customer role information related to the selected scenario from the database.

[0356] Step 3:

[0357] Based on the customer information acquired by the server, a scenario is generated from a pre-prepared template. The generated scenario data is then sent to the terminal.

[0358] Step 4:

[0359] Based on the scenario data received from the server, the terminal displays an initial message to the user, who is playing the role of a customer. The user then enters a response to the initial message.

[0360] Step 5:

[0361] User input is sent from the terminal to the server. The server analyzes the user input and generates the next customer response based on its content.

[0362] Step 6:

[0363] The server sends the next response generated by the customer role to the terminal. The terminal displays this to the user. The user then enters their response again.

[0364] Step 7:

[0365] The device uses its built-in camera and microphone to transmit the user's facial expressions and voice tone to the emotion engine in real time. The emotion engine analyzes this data to identify the user's emotional state.

[0366] Step 8:

[0367] The emotion engine recognizes the user's emotional state, which is then sent to the server. The server uses this information to adjust the customer's next response.

[0368] Step 9:

[0369] The server sends a customer's response, generated after considering emotional data, to the terminal. The terminal then displays this to the user.

[0370] Step 10:

[0371] The process from Step 5 to Step 9 is repeated, and a virtual conversation between the user and the customer progresses. The user makes closing suggestions based on the flow of the conversation and continues to input. In addition, the emotion engine monitors the user's emotional changes in real time.

[0372] Step 11:

[0373] Once the conversation ends, the terminal sends all conversation logs to the server. The server saves the conversation logs and begins analysis.

[0374] Step 12:

[0375] The server generates an evaluation and feedback on the user's closing suggestion based on the conversation log. The generated feedback is then sent to the terminal.

[0376] Step 13:

[0377] The device displays feedback received from the server to the user. The user reviews the feedback and uses it to improve their next practice session.

[0378] Through the above series of steps, users can improve their closing proposal skills through practical and emotionally responsive virtual conversations.

[0379] (Example 2)

[0380] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0381] Conventional systems for improving closing proposal skills for new employees suffer from monotonous customer responses, making it difficult to simulate the diverse reactions that occur in actual business negotiations. Furthermore, the lack of feedback that takes user emotions into account prevents effective practice that is relevant to real-world situations.

[0382] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, and an emotion analysis means. This enables the generation of diverse customer role responses that reflect the user's emotions, allowing practice in an environment close to an actual business negotiation.

[0383] "Scenario selection means" refers to a device or method that provides an interface for a user to select a practice scenario.

[0384] "Customer role information acquisition means" refers to a device or method for acquiring information about a virtual customer (customer role) from a database based on a selected scenario.

[0385] A "scenario generation means" is a device or method that generates a specific scenario by combining customer information with a pre-prepared template.

[0386] "User input receiving means" refers to a device or method for receiving content entered by a user from a terminal.

[0387] "Customer role response generation means" refers to a device or method that generates the next customer role response based on user input.

[0388] "Conversation log storage means" refers to a device or method for saving all logs of a virtual conversation.

[0389] "Feedback generation means" refers to a device or method that generates an evaluation and feedback on a user's closing suggestion based on a saved conversation log.

[0390] "Evaluation display means" refers to a device or method that displays the generated feedback on the user's terminal.

[0391] "Emotion analysis means" refers to a device or method for analyzing a user's facial expressions and tone of voice to recognize their emotions.

[0392] This invention is a system that provides a practice environment for effectively improving the closing proposal skills of new employees. The system consists of a server, terminals, and users, and their respective roles and operations are described below.

[0393] server

[0394] The server plays a central role in generating scenarios and customer responses to user input.

[0395] 1. Scenario Selection Method: The server receives scenario information specified by the user from the terminal. When the user selects "Consultation on purchasing home appliances," that information is sent to the server. The server then performs an appropriate database query based on this information.

[0396] 2. Means for obtaining customer role information: The server obtains customer role information from the database based on the selected scenario. For example, in the "consultation on purchasing home appliances" scenario, past sales history and customer attribute information are obtained using SQL queries.

[0397] 3. Scenario generation method: The server combines the acquired customer role information with templates to generate a specific scenario. For example, it creates a specific scenario such as "The customer is interested in refrigerators from companies A and B."

[0398] 4. User Input Reception Method: The server receives the information entered by the user from the terminal in real time. When the user enters "What product are you looking for?", that information is sent to the server and received by the server.

[0399] 5. Customer Role Response Generation Means: The server analyzes the received user input and generates the next customer role response based on its content. It uses a generation AI model to respond appropriately to the user's questions.

[0400] 6. Conversation log storage method: The server will save all logs of the virtual conversation. Each step of the conversation will be saved chronologically in the database for later reference.

[0401] 7. Feedback generation mechanism: The server generates an evaluation and feedback on the user's closing proposal based on the conversation log. It analyzes the log and provides feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[0402] 8. Evaluation display means: The server sends the generated feedback to the user's terminal for display.

[0403] 9. Emotion Analysis Method: The server analyzes the user's facial expressions and tone of voice to recognize their emotions. It also analyzes data obtained from the camera and microphone and incorporates it into the scenario as needed.

[0404] terminal

[0405] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[0406] 1. Scenario Selection Screen: The device will prompt the user to select a scenario on the initial screen. It will display multiple scenarios, such as "Consultation on purchasing home appliances," and wait until the user makes a selection.

[0407] 2. User Input Screen: The terminal displays an initial message for the customer role based on a scenario and accepts user input in real time. For example, it might display the message, "Hello, I'm thinking of buying a home appliance."

[0408] 3. Response Display Screen: The terminal displays the customer's response to the user's input. It displays the server's response: "I'm considering buying a refrigerator. Please tell me the differences between Company A's and Company B's products."

[0409] 4. Emotion Recognition Interface: The device uses a camera and microphone to analyze the user's facial expressions and voice tone. It captures the user's face and voice in real time and sends it to the server.

[0410] 5. Feedback screen: After the conversation ends, the device displays the feedback sent from the server. It displays the evaluation: "It was a very clear closing offer, but there was a lack of specific pricing information."

[0411] User

[0412] Users operate this system to simulate interactions with actual customers. Furthermore, an emotion engine analyzes those emotions in real time.

[0413] 1. Scenario Selection: The user selects the scenario they want to practice from the scenario selection screen on their device. Select "Consultation on purchasing home appliances" to start the scenario.

[0414] 2. Virtual Conversation Progression: The user progresses the conversation by responding to messages from the customer displayed on the terminal. For example, the user might type "What product are you looking for?" and continue the conversation with the customer.

[0415] 3. Closing Proposal: The user makes a closing proposal based on the flow of the conversation. For example, they might suggest, "Our A Company refrigerator uses the latest energy-saving technology and can reduce your electricity bill in the long run."

[0416] 4. Emotion Monitoring: The user's facial expressions and tone of voice are analyzed in real time and sent to the server. For example, if the user shows an anxious expression, this is analyzed and influences the next scenario.

[0417] 5. Receiving Feedback: After the conversation ends, the user reviews the feedback displayed on their device and uses it to improve their next practice session. For example, they might receive feedback such as, "The closing offer was very clear, but there was a lack of specific pricing information."

[0418] Usage examples and prompt messages

[0419] Usage example

[0420] The user selects the "consultation on purchasing home appliances" scenario on their device, and the conversation begins.

[0421] The first message displayed is, "Hello, I'm thinking of buying some home appliances."

[0422] The user types, "What kind of product are you looking for?", and the conversation proceeds.

[0423] The server generates a customer response saying, "I'm considering buying a refrigerator. Could you tell me the difference between products from company A and company B?" and sends it to the terminal.

[0424] A user suggests, "Our A-brand refrigerator uses the latest energy-saving technology, which will save you money on electricity bills."

[0425] Based on the conversation log and sentiment analysis results, the server generates feedback stating, "This was a very clear closing offer, but it lacked specific pricing information," and displays it on the terminal.

[0426] Example of a prompt

[0427] "Next scenario: Inquiry about purchasing home appliances. Please generate an initial message for the customer."

[0428] "Analyze the user's input and generate the next customer's response."

[0429] "Generate feedback and evaluation for the user's closing suggestion."

[0430] This system allows new employees to efficiently improve their closing proposal skills in a practical environment. Furthermore, the emotion engine enables advanced practice that takes user emotions into account.

[0431] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0432] Step 1:

[0433] The server receives scenario selection information sent from the terminal. When the user selects "consultation on purchasing home appliances" on the terminal, the server receives that information and performs the appropriate database query. The input is the user's scenario selection information, and the output is the database query related to the scenario.

[0434] Step 2:

[0435] The server retrieves customer role information from the database based on the scenario selection. It uses SQL queries to retrieve customer attribute information and past sales history related to "consultation on purchasing home appliances." The input is a database query, and the output is specific information about the customer role.

[0436] Step 3:

[0437] The server generates a scenario using a template based on the acquired customer role information. For example, it creates a specific scenario such as "The customer is interested in refrigerators from companies A and B." The input is customer role information and a template, and the output is the specific scenario.

[0438] Step 4:

[0439] The server sends the generated scenario to the user's terminal and displays the initial message of the scenario. For example, it might display the message, "Hello, I'm thinking about buying some home appliances." The input is the generated scenario, and the output is the initial message displayed on the terminal.

[0440] Step 5:

[0441] The terminal accepts user input in real time. When a user types "What product are you looking for?", that information is sent to the server. The input is the user's message, and the output is the input data sent to the server.

[0442] Step 6:

[0443] The server analyzes user input and generates the next customer response based on that input. It uses a generative AI model to generate appropriate responses to user questions. For example, it might generate the response, "I'm looking at refrigerators. Could you tell me the differences between products from company A and company B?" The input is the user's input data, and the output is the generated customer response.

[0444] Step 7:

[0445] The server sends the generated customer response to the user's terminal for display. The terminal displays the content and prompts the user for the next action. The input is the generated customer response, and the output is the message displayed on the terminal.

[0446] Step 8:

[0447] The server saves logs of all virtual conversations. Each step of the conversation is saved chronologically in the database for later reference. The input is the conversation log data, and the output is the saved conversation log.

[0448] Step 9:

[0449] The server generates an evaluation and feedback on the user's closing proposal based on the conversation log. It analyzes the log and provides feedback such as, "This was a very clear closing proposal, but it lacked specific pricing." The input is the conversation log data, and the output is the feedback content.

[0450] Step 10:

[0451] The server sends the generated feedback to the user's terminal, which then displays the feedback. The input is the generated feedback, and the output is the evaluation and suggestions displayed on the terminal.

[0452] Step 11:

[0453] The device uses a camera and microphone to analyze the user's facial expressions and voice tone. It captures the user's face and voice in real time and sends it to the server. The input is data obtained from the camera and microphone, and the output is the analyzed data sent to the server.

[0454] Step 12:

[0455] The server uses emotion analysis tools to analyze the user's facial expressions and tone of voice, and recognizes their emotions. For example, if it detects an anxious facial expression or tone of voice, it makes positive changes to the next scenario based on the analysis results. The input is the analysis data sent from the terminal, and the output is the analysis results and the modified scenario.

[0456] (Application Example 2)

[0457] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0458] Traditional customer service training systems have made it difficult for new sales staff to effectively learn how to respond to real-world customer service scenarios. Furthermore, it has been challenging to conduct more realistic training by analyzing user emotions in real time and providing feedback based on that analysis. Therefore, there is a need for a system that can quickly and reliably improve the customer service skills of new sales staff.

[0459] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0460] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, an emotion recognition means that analyzes emotions in real time using a camera and a microphone, a means that transmits information including the analyzed emotion data to the server, and a means that outputs the customer role's response as voice by speech synthesis. This makes it possible for new sales staff to practice customer service skills in a realistic manner and to effectively improve their skills through feedback that includes real-time emotion analysis.

[0461] A "scenario selection method" is a means for a user to select a specific scenario to use in training.

[0462] The "customer role information acquisition method" is a means of acquiring relevant customer role information from a database based on the selected scenario.

[0463] A "scenario generation method" is a method for generating a scenario from a template based on acquired customer role information.

[0464] "User input receiving means" refers to means for receiving user input.

[0465] "Customer role response generation means" refers to means for generating the next response of the customer role based on user input.

[0466] A "conversation log saving method" is a means of saving logs of virtual conversations between a user and a customer.

[0467] A "feedback generation method" is a means of generating evaluations and feedback on a user's closing suggestion based on conversation logs.

[0468] "Evaluation display means" refers to a means of sending the generated feedback to a terminal and displaying it to the user.

[0469] "Emotion recognition means" refers to a method of analyzing a user's emotions in real time using a camera and microphone.

[0470] "Means for transmitting information including analyzed emotion data to a server" refers to means for transmitting emotion data analyzed by emotion recognition means to a server.

[0471] "Means for outputting customer responses as audio using speech synthesis" refers to means for outputting the generated customer responses as audio.

[0472] The system for realizing this invention consists of three components: a server, a terminal, and a user. The functions and roles of each component are described below.

[0473] server

[0474] The server plays a central role in generating and managing scenarios, and receiving and analyzing user input. The server includes the following:

[0475] 1. Scenario selection method:

[0476] When a user selects a specific scenario to use for training, the scenario selection mechanism receives that information.

[0477] 2. Means of obtaining customer role information:

[0478] Based on the selected scenario, relevant customer information is retrieved from the database. For example, in a scenario involving a consultation about purchasing home appliances, past purchase history and customer attribute information are retrieved.

[0479] 3. Scenario generation method:

[0480] Based on the acquired customer information, a scenario is generated from a template. This scenario includes instructions on how the user should respond.

[0481] 4. User input receiving means:

[0482] It receives user input in real time. For example, it receives information such as "What kind of product are you looking for?" entered by the user.

[0483] 5. Means for generating customer role responses:

[0484] Based on user input, the system generates the next response from the customer. For example, a response like, "I'm looking to buy a refrigerator. Could you tell me the difference between products from company A and company B?"

[0485] 6. Means of saving conversation logs:

[0486] The system saves a chronological log of virtual conversations between the user and the customer. This serves as reference information when generating feedback later.

[0487] 7. Feedback generation means:

[0488] Based on saved conversation logs, the system generates evaluations and feedback on the user's closing proposal. For example, it might provide feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[0489] 8. Means of displaying evaluations:

[0490] The feedback generated by the feedback generation mechanism is sent to the terminal and displayed to the user.

[0491] 9. Emotion recognition means:

[0492] The system uses a camera and microphone to analyze the user's facial expressions and voice tone in real time. For example, if the user shows an expression of dissatisfaction, the system analyzes that emotional data.

[0493] 10. Means for transmitting information, including analyzed sentiment data, to a server:

[0494] The emotion data analyzed by the emotion recognition system is sent to the server.

[0495] 11. Means for outputting the customer's response as audio using speech synthesis:

[0496] The generated customer responses are output as audio. This allows users to receive training in a more realistic way.

[0497] terminal

[0498] The terminal provides an interface for the user to communicate with the server and engage in virtual conversations. It includes the following functions:

[0499] 1. Scenario selection screen:

[0500] The user is prompted to select a scenario on the initial screen.

[0501] 2. User input screen:

[0502] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[0503] 3. Response display screen:

[0504] Displays the customer's response to the user's input.

[0505] 4. Emotion Recognition Interface:

[0506] The device uses cameras and microphones to analyze the user's facial expressions and voice tone.

[0507] 5. Feedback screen:

[0508] After the conversation ends, the feedback sent from the server will be displayed.

[0509] User

[0510] Users operate this system to simulate interactions with actual customers.

[0511] 1. Scenario Selection:

[0512] Select the scenario you want to practice on the device's scenario selection screen. For example, select "Consultation on purchasing home appliances."

[0513] 2. The progress of the virtual conversation:

[0514] Respond to messages from the customer displayed on the terminal and advance the conversation.

[0515] 3. Closing proposal:

[0516] Based on the flow of the conversation, make a closing proposal. For example, you might suggest, "Our A-brand refrigerators use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[0517] 4. Emotional monitoring:

[0518] During the conversation, the user's facial expressions and voice tone are analyzed in real time, and that data is sent to the server.

[0519] 5. Feedback received:

[0520] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[0521] Specific example:

[0522] Examples of prompts to input into a generative AI model include the following:

[0523] Please select the "Consultation on purchasing home appliances" scenario. The initial message from the customer is "Hello, I'm thinking about purchasing home appliances." Next, respond with "What kind of products are you looking for?" If the user's expression seems anxious, make the customer's response more positive. Finally, make a closing suggestion by saying, "Here are some products I recommend."

[0524] In this way, by using this system, new sales staff can efficiently improve their customer service skills in a practical environment.

[0525] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0526] Step 1:

[0527] Scenario Selection

[0528] The server receives scenario information specified by the user via a scenario selection mechanism. The user selects the scenario they want to practice (for example, "consultation on purchasing home appliances") on the terminal's scenario selection screen. The input is scenario information, and the output is the system settings based on the selected scenario.

[0529] Step 2:

[0530] Acquisition of customer role information

[0531] The server uses a customer role information acquisition mechanism to retrieve relevant customer role information from the database based on the selected scenario. For example, past purchase history and attribute information of a customer considering purchasing home appliances. The input is a request for customer role information based on the scenario, and the output is the retrieved customer role information.

[0532] Step 3:

[0533] Scenario generation

[0534] The server generates a scenario from a template based on customer role information obtained via a scenario generation mechanism. For example, it might generate a scenario that includes the initial message, "Hello, I'm thinking of buying some home appliances." The input is customer role information, and the output is the generated scenario.

[0535] Step 4:

[0536] Receiving user input

[0537] The terminal displays customer-facing messages based on a scenario and accepts user input in real time. For example, the user might input, "What product are you looking for?" The input is the user's response, and the output is a record of that response.

[0538] Step 5:

[0539] Customer response generation

[0540] The server uses a customer response generation mechanism to analyze user input and generate the next customer response based on that input. For example, it might generate a response like, "I'm looking at refrigerators. Could you tell me the difference between products from company A and company B?" The input is the user's response, and the output is the generated customer response.

[0541] Step 6:

[0542] Saving conversation logs

[0543] The server uses a conversation log storage mechanism to save logs of virtual conversations between the user and the customer. The saved content includes the user's statements, the customer's generated responses, and timestamps. The input is each sequence of the conversation, and the output is the saved log.

[0544] Step 7:

[0545] Emotion analysis and transmission

[0546] The device uses a camera and microphone to capture the user's facial expressions and voice tone in real time, and analyzes them using emotion recognition technology. The analyzed emotion data is sent to a server. The input is the user's facial expressions and voice, and the output is the analyzed emotion data.

[0547] Step 8:

[0548] Audio output of the customer's response

[0549] The server outputs the customer's responses, generated using speech synthesis, as audio. This allows the user to train in more realistic scenarios. The input is the generated customer's responses, and the output is the audio response.

[0550] Step 9:

[0551] Generating and displaying feedback

[0552] The server generates an evaluation and feedback on the user's closing proposal using a feedback generation mechanism, based on the conversation log and analyzed sentiment data. This is then sent to the terminal using an evaluation display mechanism and displayed to the user. For example, feedback such as "This was a very clear closing proposal, but it lacked specific pricing" might be displayed. The input is the conversation log and sentiment data, and the output is the generated feedback.

[0553] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0554] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0555] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0556] [Second Embodiment]

[0557] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0558] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0559] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0560] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0561] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0562] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0563] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0564] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0565] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0566] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0567] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0568] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0569] The system of this invention is designed to provide a practice environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users (new employees), and its specific operation is described below.

[0570] server

[0571] The server plays a central role in generating scenarios and customer responses.

[0572] 1. Scenario selection method:

[0573] The server receives the scenario selection information specified by the user.

[0574] Specific example: The user selects the scenario "consultation on purchasing home appliances."

[0575] 2. Means of obtaining customer role information:

[0576] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[0577] Specific example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[0578] 3. Scenario generation method:

[0579] Based on the acquired customer information, a scenario is generated from a pre-prepared template.

[0580] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[0581] 4. User input receiving means:

[0582] The server receives user input from the terminal.

[0583] Specific example: Receive information entered by the user, such as "What kind of product are you looking for?".

[0584] 5. Means for generating customer role responses:

[0585] The server analyzes the received user input and generates the next response from the customer based on that input.

[0586] Specific example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[0587] 6. Means of saving conversation logs:

[0588] The server saves logs of virtual conversations between users and customers.

[0589] Specific example: Save each exchange in a conversation in chronological order.

[0590] 7. Feedback generation means:

[0591] Based on the conversation log, it generates an evaluation and feedback on the user's closing suggestion.

[0592] Specific example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[0593] 8. Means of displaying evaluations:

[0594] The server sends the generated feedback to the terminal and displays it to the user.

[0595] Specific example: Display the feedback content on the user's device.

[0596] terminal

[0597] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[0598] 1. Scenario selection screen:

[0599] The user is prompted to select a scenario on the initial screen.

[0600] Specific example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[0601] 2. User input screen:

[0602] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[0603] Specific example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[0604] 3. Response display screen:

[0605] Displays the customer's response to the user's input.

[0606] Specific example: The server displays the response, "I am considering purchasing a refrigerator. Please tell me the differences between products from company A and company B."

[0607] 4. Feedback screen:

[0608] After the conversation ends, the feedback sent from the server will be displayed.

[0609] Specific example: Display a review stating, "The closing proposal was very clear, but it lacked a specific price."

[0610] User

[0611] Users operate this system to simulate interactions with actual customers.

[0612] 1. Scenario Selection:

[0613] The user selects the scenario they want to practice from the scenario selection screen on their device.

[0614] Example: Select "Consultation on purchasing home appliances".

[0615] 2. The progress of the virtual conversation:

[0616] The user progresses the conversation by responding to messages from a customer displayed on their device.

[0617] Example: Continue the conversation with the customer by typing, "What kind of product are you looking for?"

[0618] 3. Closing proposal:

[0619] Based on the flow of the conversation, the user makes a closing proposal.

[0620] Specific example: "Our refrigerators from Company A use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[0621] 4. Feedback received:

[0622] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[0623] Specific example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[0624] In this way, by using this system, new employees can efficiently improve their closing proposal skills in a practical environment.

[0625] The following describes the processing flow.

[0626] Step 1:

[0627] The user opens the scenario selection screen using their device and specifies the scenario they want to practice. The scenario selection information is sent to the server.

[0628] Step 2:

[0629] The server analyzes the scenario selection information received from the user and retrieves customer role information related to the selected scenario from the database.

[0630] Step 3:

[0631] Based on the customer information acquired by the server, a scenario is generated from a pre-prepared template. The generated scenario data is then sent to the terminal.

[0632] Step 4:

[0633] Based on the scenario data received from the server, the terminal displays an initial message to the user, who is playing the role of a customer. The user then enters a response to the initial message.

[0634] Step 5:

[0635] User input is sent from the terminal to the server. The server analyzes the user input and generates the next customer response based on its content.

[0636] Step 6:

[0637] The server sends the next response generated by the customer role to the terminal. The terminal displays this to the user. The user then enters their response again.

[0638] Step 7:

[0639] Steps 5 and 6 are repeated, and a virtual conversation between the user and the customer progresses. The user makes closing suggestions based on the flow of the conversation and continues to input.

[0640] Step 8:

[0641] Once the conversation ends, the terminal sends all conversation logs to the server. The server saves the conversation logs and begins analysis.

[0642] Step 9:

[0643] The server generates an evaluation and feedback on the user's closing suggestion based on the conversation log. The generated feedback is then sent to the terminal.

[0644] Step 10:

[0645] The device displays feedback received from the server to the user. The user reviews the feedback and uses it to improve their next practice session.

[0646] (Example 1)

[0647] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0648] Providing an effective training environment for new employees to practically improve their closing proposal skills is challenging. Traditional training lacks real-time feedback and practice using concrete scenarios, hindering the development of skills relevant to actual work. To address this challenge, a system is needed that provides real-time quantitative and qualitative feedback through virtual customer interactions.

[0649] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0650] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a user input means via a terminal, a scenario display means via a terminal, a customer role response display means via a terminal, a feedback display means via a terminal, a means for acquiring customer role information from a database via the server, a scenario generation means via the server, a user input analysis means via the server, a customer role message display means via a terminal, and a means for generating and displaying evaluations and feedback based on the conversation log. This enables the user to effectively practice dialogue skills and closing proposal skills through real-time virtual conversations based on scenarios and to receive detailed feedback.

[0651] A "scenario selection method" is a function that provides an interface for users to select the scenario they want to practice.

[0652] The "customer role information acquisition method" is a function that retrieves information about customer roles related to a scenario from a database.

[0653] The "scenario generation method" is a function that generates a scenario using a template based on the acquired customer role information.

[0654] A "user input receiving means" is a function that receives input from the user and analyzes it.

[0655] The "customer role response generation means" is a function that generates the next response of the customer role based on user input.

[0656] The "conversation log saving method" is a function that saves a chronological log of a virtual conversation between the user and the customer.

[0657] The "feedback generation method" is a function that generates evaluations and feedback on the user's closing suggestion based on the saved conversation log.

[0658] "Evaluation display means" refers to a function that displays the generated feedback to the user.

[0659] "User input means via terminal" refers to a function that accepts user input in real time.

[0660] "Scenario display means via terminal" refers to a function that displays the generated scenario to the user.

[0661] "Means for displaying customer responses via a terminal" refers to a function that displays the customer's response to user input.

[0662] "Means of displaying feedback via the terminal" refers to a function that displays the generated feedback to the user after the conversation has ended.

[0663] "Means of retrieving customer role information from the database by the server" refers to the function of retrieving customer role information from the database.

[0664] "Server-based scenario generation method" refers to a function that generates scenarios using templates based on customer information.

[0665] "Server-based user input analysis means" refers to a function that analyzes user input and generates the next customer response based on that analysis.

[0666] "Means for displaying customer-role messages via a terminal" refers to a function that displays messages from the customer role to the user.

[0667] "Means for generating and displaying evaluations and feedback based on conversation logs" refers to a function that generates evaluations and feedback on a user's closing suggestion based on saved conversation logs and displays them to the user.

[0668] The present invention provides a training environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users, and each component works in cooperation with the others.

[0669] System Configuration

[0670] server

[0671] The server plays a central role in the system, providing the means to generate scenarios and customer responses. The server has the following functions:

[0672] 1. Scenario Selection Method

[0673] The server receives scenario selection information specified by the user.

[0674] Example: The user selects the scenario "Consultation on purchasing home appliances".

[0675] 2. Means for obtaining customer role information

[0676] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[0677] Example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[0678] 3. Scenario generation means

[0679] The server generates a scenario from a template based on the customer role information it has acquired.

[0680] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[0681] 4. User input receiving means

[0682] The server receives user input from the terminal.

[0683] Example: Receive information entered by the user, such as "What kind of product are you looking for?".

[0684] 5. Customer response generation means

[0685] The server analyzes the user's input and generates the next customer's response based on that input.

[0686] Example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[0687] 6. Means of saving conversation logs

[0688] The server saves logs of virtual conversations between users and customers.

[0689] Example: Save all conversations in chronological order.

[0690] 7. Feedback generation means

[0691] The server generates an evaluation and feedback on the user's closing suggestion based on the saved conversation logs.

[0692] Example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[0693] 8. Evaluation display means

[0694] The server sends the generated feedback to the terminal and displays it to the user.

[0695] Example: Display the feedback content on the user's device.

[0696] terminal

[0697] The terminal provides an interface for users to engage in virtual conversations through communication with the server. The terminal's functions are as follows:

[0698] 1. Scenario Selection Screen

[0699] The user is prompted to select a scenario on the initial screen.

[0700] Example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[0701] 2. User input screen

[0702] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[0703] Example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[0704] 3. Response display screen

[0705] Displays the customer's response to the user's input.

[0706] Example: The server will display the response, "I am considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[0707] 4. Feedback screen

[0708] After the conversation ends, the feedback sent from the server will be displayed.

[0709] Example: Display a review such as, "It was a very clear closing proposal, but it lacked a specific price."

[0710] User

[0711] The user operates this system to simulate interactions with actual customers. Follow these steps:

[0712] 1. Scenario Selection

[0713] The user selects the scenario they want to practice from the scenario selection screen on their device.

[0714] Example: Select "Consultation on purchasing home appliances".

[0715] 2. Progress of the virtual conversation

[0716] The user progresses the conversation by responding to messages from a customer displayed on their device.

[0717] Example: Continue the conversation with the customer by typing "What kind of product are you looking for?"

[0718] 3. Closed proposal

[0719] Based on the flow of the conversation, the user makes a closing proposal.

[0720] Example: "Our A Company refrigerators use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[0721] 4. Receiving Feedback

[0722] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[0723] Example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[0724] Example of a prompt

[0725] "Analyze the following user input and generate an appropriate response to the virtual customer's question: 'What products are you looking for?'"

[0726] In this way, by using this system, new employees can efficiently improve their closing proposal skills in a practical environment.

[0727] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0728] Step 1: Scenario Selection

[0729] The server receives scenario selection information sent from the terminal. Based on this input information, the server recognizes the selected scenario and prepares for the next processing. For example, if a user selects the scenario "consultation on purchasing home appliances," that information is sent to the server.

[0730] Step 2: Obtain customer role information

[0731] The server uses the scenario selection information to retrieve relevant customer role information from the database. This process involves executing database queries and extracting the necessary data. The retrieved data includes customer attribute information and past sales history. For example, the server retrieves attribute information and past sales history for a "customer considering purchasing home appliances."

[0732] Step 3: Scenario Generation

[0733] The server generates a scenario using a template based on the customer role information it has acquired. In this process, data is applied to the template to generate a specific conversation scenario. The generated scenario includes an initial message and questions. For example, the server generates a scenario based on "consultation about purchasing home appliances" and creates an initial message such as "Hello, I'm thinking about purchasing home appliances."

[0734] Step 4: Display the scenario

[0735] The terminal displays the scenario sent from the server to the user. In this process, the terminal converts the received data into an appropriate layout for display on the screen. It receives scenario data from the server as input and displays it to the user as output. For example, the terminal displays the message "Hello, I'm thinking about buying home appliances" to the user.

[0736] Step 5: User response input

[0737] The user enters a response to a customer-facing message displayed on the terminal. The entered data is sent from the terminal to the server. For example, the user might enter "What product are you looking for?"

[0738] Step 6: Receiving and analyzing user input

[0739] The server analyzes user input received from the terminal. This process uses natural language processing techniques to understand the user's intent and generate the next customer response. It receives the user's message as input, analyzes it, and then generates the next response data. For example, the server analyzes the input "What kind of product are you looking for?" and generates the response "I'm considering a refrigerator. Could you tell me the difference between products from company A and company B?"

[0740] Step 7: Customer's reaction

[0741] The terminal displays customer responses generated by the server to the user. In this process, it receives response data from the server and displays it appropriately on the screen. It receives response data as input and displays it to the user as output. For example, the terminal displays, "I'm considering buying a refrigerator. Could you tell me the differences between products from company A and company B?"

[0742] Step 8: Saving the conversation log

[0743] The server saves logs of virtual conversations between the user and the customer. This process stores all conversational exchanges chronologically in a database. It takes conversational data as input and saves it to the database as output. For example, the server saves each individual exchange in a conversation as a log.

[0744] Step 9: Generating Feedback

[0745] The server generates an evaluation and feedback on the user's closing proposal based on the saved conversation logs. This process analyzes the conversation logs and creates feedback based on evaluation criteria. It takes conversation logs as input and generates feedback data as output. For example, the server might generate feedback such as, "This was a very clear closing proposal, but it lacked a specific price offer."

[0746] Step 10: Displaying Feedback

[0747] The terminal displays feedback sent from the server to the user. This process converts the feedback data into an appropriate layout for display on the screen. It receives feedback data as input and displays it to the user as output. For example, the terminal displays the feedback, "It was a very clear closing offer, but it lacked specific pricing information."

[0748] (Application Example 1)

[0749] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0750] For new employees to efficiently improve skills such as presentations and content proposals, practice in a realistic environment is necessary. However, many companies have limited employees and resources for such practice, making it difficult to provide effective training. Furthermore, there are challenges such as limited opportunities for individual feedback and difficulty in identifying areas for improvement through self-assessment.

[0751] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0752] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a response training means including scenario selection information, and a prompt sentence generation means. This allows the user to practice in real time through interaction with a virtual customer and receive individualized feedback.

[0753] "Scenario selection method" refers to the means by which users select a scenario for practice.

[0754] "Method for obtaining customer role information" refers to a means for obtaining information about customer roles related to the selected scenario from a database.

[0755] A "scenario generation method" is a means of generating a scenario from a pre-prepared template based on acquired customer role information.

[0756] "User input receiving means" refers to means for receiving user input.

[0757] "Customer role response generation means" refers to means for generating the next response of the customer role based on the received user input.

[0758] A "conversation log saving method" is a means for saving logs of virtual conversations between a user and a customer.

[0759] A "feedback generation method" is a means of generating evaluations and feedback on a user's closing suggestion based on conversation logs.

[0760] "Evaluation display means" refers to a means for displaying the generated feedback to the user.

[0761] A "response training method including scenario selection information" is a means for training responses based on scenario selection information specified by the user.

[0762] A "prompt message generation means" is a means for generating prompt messages that users will use.

[0763] The present invention is a system for efficiently improving the presentation skills and content proposal abilities of new employees. This system includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a response training means including scenario selection information, and a prompt sentence generation means.

[0764] server

[0765] Scenario selection method

[0766] The server receives information from the user about the scenario they want to practice. For example, if the user selects "New Product Presentation Scenario," the server processes this information.

[0767] Customer information acquisition method

[0768] The server retrieves customer role information from the database based on the selected scenario. This information includes customer profiles and past conversation history.

[0769] Scenario generation method

[0770] Based on the customer information obtained, the server generates an appropriate scenario from pre-prepared templates. For example, it might generate a "presentation scenario emphasizing the advantages of a new product."

[0771] User input receiving means

[0772] The server receives user input (for example, a question like, "How is the new product better?") in real time.

[0773] Customer response generation means

[0774] The server analyzes user input and generates the next response from the customer role based on that input. For example, in response to user input, it might generate a response such as, "The new product incorporates the latest technology and is very easy to use."

[0775] Conversation log saving method

[0776] The server stores a chronological log of the conversation between the user and the customer. This log is used later to generate feedback.

[0777] Feedback generation means

[0778] Based on the conversation log, the server generates user closing suggestions and an overall evaluation and feedback on the presentation. For example, it might provide feedback such as, "The presentation was generally easy to understand, but it lacked detailed explanations of the specifications."

[0779] Evaluation display means

[0780] The server sends the generated feedback to the user's device, allowing the user to check their performance.

[0781] terminal

[0782] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[0783] Scenario selection screen

[0784] The initial screen allows the user to select a scenario. For example, a screen might be displayed where the user can select "New Product Presentation Scenario."

[0785] User input screen

[0786] It displays an initial message from a customer based on a scenario and accepts user input in real time. For example, it might display the message, "Hello, please give me a presentation on the new product," and then receive a response from the user.

[0787] Response display screen

[0788] It displays the customer's response to the user's input. For example, it might display the server's response: "The new product incorporates the latest technology and is very easy to use."

[0789] Feedback screen

[0790] After the conversation ends, the server displays the feedback it has received. For example, it might show an evaluation such as, "The presentation was generally easy to understand, but it lacked detailed explanations of the specifications."

[0791] User

[0792] Scenario Selection

[0793] The user selects the scenario they want to practice from the scenario selection screen on their device. For example, they might select "New Product Presentation Scenario."

[0794] Progress of the virtual conversation

[0795] The user progresses the conversation by responding to messages from a customer displayed on the device. For example, they might type, "How is the new product better?" to continue the conversation.

[0796] Closed proposal

[0797] Based on the flow of the conversation, the user makes a closing suggestion. For example, they might suggest, "This new product is reasonably priced, so please give it a try."

[0798] Feedback received

[0799] After the conversation ends, review the feedback displayed on the device and use it to improve your next practice session. For example, you might receive feedback such as, "The presentation was generally easy to understand, but there was a lack of detailed explanation of the specifications," and use that to identify areas for improvement.

[0800] Specific examples and prompt statements

[0801] This example illustrates how new content creators can learn to present new content. An example of a prompt is: "Our new content offers clear visuals and intuitive controls that will excite viewers. Is there anything specific you're concerned about?" In this way, newcomers can effectively improve their skills by utilizing scenarios provided by the server and real-time feedback.

[0802] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0803] Step 1:

[0804] The server receives scenario selection information from the user through a scenario selection mechanism. Specifically, the user selects the scenario they want to practice and sends that information to the server. The input is scenario selection information, and the output is the selected scenario information.

[0805] Step 2:

[0806] The server uses a customer role information acquisition mechanism to retrieve customer role information related to the selected scenario from the database. This includes data such as the customer role's profile and past conversation history. The input is the selected scenario information, and the output is the retrieved customer role information.

[0807] Step 3:

[0808] The server uses a scenario generation mechanism to generate a scenario based on the acquired customer role information. A specific scenario is created using a template. The input is customer role information and a template, and the output is the generated scenario.

[0809] Step 4:

[0810] The server receives user input through a user input receiving mechanism. The user inputs questions and suggestions for the customer role based on a scenario. The input is direct information from the user, and the output is that same input information.

[0811] Step 5:

[0812] The server uses a customer role response generation mechanism to analyze the received user input and generates the next customer role response using a generation AI model. The input is the user's input information, and the output is the generated customer role response.

[0813] Step 6:

[0814] The server uses a conversation log storage mechanism to save logs of virtual conversations between the user and the customer. The input consists of the user's input information and the customer's responses, and the output is the saved conversation log.

[0815] Step 7:

[0816] The server uses a feedback generation mechanism to generate an evaluation and feedback on the user's closing suggestion based on the saved conversation log. The input is the saved conversation log, and the output is the generated feedback information.

[0817] Step 8:

[0818] The server displays the generated feedback to the user through the evaluation display means. The feedback content is displayed on the user's terminal. The input is the feedback information, and the output is the feedback displayed on the user's terminal.

[0819] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0820] The system of this invention is designed to provide a practice environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users (new employees), and its specific operation is described below.

[0821] server

[0822] The server plays a central role in generating scenarios and customer responses. It also uses an emotion engine to analyze user emotions and reflect them in the system's responses.

[0823] 1. Scenario selection method:

[0824] The server receives the scenario selection information specified by the user.

[0825] Specific example: The user selects the scenario "consultation on purchasing home appliances."

[0826] 2. Means of obtaining customer role information:

[0827] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[0828] Specific example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[0829] 3. Scenario generation method:

[0830] Based on the acquired customer information, a scenario is generated from a pre-prepared template.

[0831] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[0832] 4. User input receiving means:

[0833] The server receives user input from the terminal.

[0834] Specific example: Receive information entered by the user, such as "What kind of product are you looking for?".

[0835] 5. Means for generating customer role responses:

[0836] The server analyzes the received user input and generates the next response from the customer based on that input.

[0837] Specific example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[0838] 6. Means of saving conversation logs:

[0839] The server saves logs of virtual conversations between users and customers.

[0840] Specific example: Save each exchange in a conversation in chronological order.

[0841] 7. Feedback generation means:

[0842] Based on the conversation log, it generates an evaluation and feedback on the user's closing suggestion.

[0843] Specific example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[0844] 8. Means of displaying evaluations:

[0845] The server sends the generated feedback to the terminal and displays it to the user.

[0846] Specific example: Display the feedback content on the user's device.

[0847] 9. Emotional Engine:

[0848] The server uses an emotion engine to analyze the user's facial expressions and tone of voice to recognize their emotions.

[0849] Specific example: If a user shows an expression of dissatisfaction, analyze that emotional data and generate a scenario that includes positive information.

[0850] terminal

[0851] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[0852] 1. Scenario selection screen:

[0853] The user is prompted to select a scenario on the initial screen.

[0854] Specific example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[0855] 2. User input screen:

[0856] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[0857] Specific example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[0858] 3. Response display screen:

[0859] Displays the customer's response to the user's input.

[0860] Specific example: The server displays the response, "I am considering purchasing a refrigerator. Please tell me the differences between products from company A and company B."

[0861] 4. Emotion Recognition Interface:

[0862] The device uses cameras and microphones to analyze the user's facial expressions and voice tone.

[0863] Specific example: Sending the tone of voice and facial expressions of a user to the server in real time when they speak.

[0864] 5. Feedback screen:

[0865] After the conversation ends, the feedback sent from the server will be displayed.

[0866] Specific example: Display a review stating, "The closing proposal was very clear, but it lacked a specific price."

[0867] User

[0868] Users operate this system to simulate interactions with actual customers. Furthermore, an emotion engine analyzes those emotions in real time.

[0869] 1. Scenario Selection:

[0870] The user selects the scenario they want to practice from the scenario selection screen on their device.

[0871] Example: Select "Consultation on purchasing home appliances".

[0872] 2. The progress of the virtual conversation:

[0873] The user progresses the conversation by responding to messages from a customer displayed on their device.

[0874] Example: Continue the conversation with the customer by typing, "What kind of product are you looking for?"

[0875] 3. Closing proposal:

[0876] Based on the flow of the conversation, the user makes a closing proposal.

[0877] Specific example: "Our refrigerators from Company A use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[0878] 4. Emotional monitoring:

[0879] During the conversation, the user's facial expressions and voice tone are analyzed in real time, and that data is sent to the server.

[0880] Specific example: If a user shows signs of anxiety, this is analyzed and influences the next scenario.

[0881] 5. Feedback received:

[0882] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[0883] Specific example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[0884] Thus, by using this system, new employees can not only efficiently improve their closing proposal skills in a practical environment, but also engage in more advanced practice that takes user emotions into consideration.

[0885] The following describes the processing flow.

[0886] Step 1:

[0887] The user opens the scenario selection screen using their device and specifies the scenario they want to practice. The scenario selection information is sent to the server.

[0888] Step 2:

[0889] The server analyzes the scenario selection information received from the user and retrieves customer role information related to the selected scenario from the database.

[0890] Step 3:

[0891] Based on the customer information acquired by the server, a scenario is generated from a pre-prepared template. The generated scenario data is then sent to the terminal.

[0892] Step 4:

[0893] Based on the scenario data received from the server, the terminal displays an initial message to the user, who is playing the role of a customer. The user then enters a response to the initial message.

[0894] Step 5:

[0895] User input is sent from the terminal to the server. The server analyzes the user input and generates the next customer response based on its content.

[0896] Step 6:

[0897] The server sends the next response generated by the customer role to the terminal. The terminal displays this to the user. The user then enters their response again.

[0898] Step 7:

[0899] The device uses its built-in camera and microphone to transmit the user's facial expressions and voice tone to the emotion engine in real time. The emotion engine analyzes this data to identify the user's emotional state.

[0900] Step 8:

[0901] The emotion engine recognizes the user's emotional state, which is then sent to the server. The server uses this information to adjust the customer's next response.

[0902] Step 9:

[0903] The server sends a customer's response, generated after considering emotional data, to the terminal. The terminal then displays this to the user.

[0904] Step 10:

[0905] The process from Step 5 to Step 9 is repeated, and a virtual conversation between the user and the customer progresses. The user makes closing suggestions based on the flow of the conversation and continues to input. In addition, the emotion engine monitors the user's emotional changes in real time.

[0906] Step 11:

[0907] Once the conversation ends, the terminal sends all conversation logs to the server. The server saves the conversation logs and begins analysis.

[0908] Step 12:

[0909] The server generates an evaluation and feedback on the user's closing suggestion based on the conversation log. The generated feedback is then sent to the terminal.

[0910] Step 13:

[0911] The device displays feedback received from the server to the user. The user reviews the feedback and uses it to improve their next practice session.

[0912] Through the above series of steps, users can improve their closing proposal skills through practical and emotionally responsive virtual conversations.

[0913] (Example 2)

[0914] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0915] Conventional systems for improving closing proposal skills for new employees suffer from monotonous customer responses, making it difficult to simulate the diverse reactions that occur in actual business negotiations. Furthermore, the lack of feedback that takes user emotions into account prevents effective practice that is relevant to real-world situations.

[0916] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, and an emotion analysis means. This enables the generation of diverse customer role responses that reflect the user's emotions, allowing practice in an environment close to an actual business negotiation.

[0917] "Scenario selection means" refers to a device or method that provides an interface for a user to select a practice scenario.

[0918] "Customer role information acquisition means" refers to a device or method for acquiring information about a virtual customer (customer role) from a database based on a selected scenario.

[0919] A "scenario generation means" is a device or method that generates a specific scenario by combining customer information with a pre-prepared template.

[0920] "User input receiving means" refers to a device or method for receiving content entered by a user from a terminal.

[0921] "Customer role response generation means" refers to a device or method that generates the next customer role response based on user input.

[0922] "Conversation log storage means" refers to a device or method for saving all logs of a virtual conversation.

[0923] "Feedback generation means" refers to a device or method that generates an evaluation and feedback on a user's closing suggestion based on a saved conversation log.

[0924] "Evaluation display means" refers to a device or method that displays the generated feedback on the user's terminal.

[0925] "Emotion analysis means" refers to a device or method for analyzing a user's facial expressions and tone of voice to recognize their emotions.

[0926] This invention is a system that provides a practice environment for effectively improving the closing proposal skills of new employees. The system consists of a server, terminals, and users, and their respective roles and operations are described below.

[0927] server

[0928] The server plays a central role in generating scenarios and customer responses to user input.

[0929] 1. Scenario Selection Method: The server receives scenario information specified by the user from the terminal. When the user selects "Consultation on purchasing home appliances," that information is sent to the server. The server then performs an appropriate database query based on this information.

[0930] 2. Means for obtaining customer role information: The server obtains customer role information from the database based on the selected scenario. For example, in the "consultation on purchasing home appliances" scenario, past sales history and customer attribute information are obtained using SQL queries.

[0931] 3. Scenario generation method: The server combines the acquired customer role information with templates to generate a specific scenario. For example, it creates a specific scenario such as "The customer is interested in refrigerators from companies A and B."

[0932] 4. User Input Reception Method: The server receives the information entered by the user from the terminal in real time. When the user enters "What product are you looking for?", that information is sent to the server and received by the server.

[0933] 5. Customer Role Response Generation Means: The server analyzes the received user input and generates the next customer role response based on its content. It uses a generation AI model to respond appropriately to the user's questions.

[0934] 6. Conversation log storage method: The server will save all logs of the virtual conversation. Each step of the conversation will be saved chronologically in the database for later reference.

[0935] 7. Feedback generation mechanism: The server generates an evaluation and feedback on the user's closing proposal based on the conversation log. It analyzes the log and provides feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[0936] 8. Evaluation display means: The server sends the generated feedback to the user's terminal for display.

[0937] 9. Emotion Analysis Method: The server analyzes the user's facial expressions and tone of voice to recognize their emotions. It also analyzes data obtained from the camera and microphone and incorporates it into the scenario as needed.

[0938] terminal

[0939] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[0940] 1. Scenario Selection Screen: The device will prompt the user to select a scenario on the initial screen. It will display multiple scenarios, such as "Consultation on purchasing home appliances," and wait until the user makes a selection.

[0941] 2. User Input Screen: The terminal displays an initial message for the customer role based on a scenario and accepts user input in real time. For example, it might display the message, "Hello, I'm thinking of buying a home appliance."

[0942] 3. Response Display Screen: The terminal displays the customer's response to the user's input. It displays the server's response: "I'm considering buying a refrigerator. Please tell me the differences between Company A's and Company B's products."

[0943] 4. Emotion Recognition Interface: The device uses a camera and microphone to analyze the user's facial expressions and voice tone. It captures the user's face and voice in real time and sends it to the server.

[0944] 5. Feedback screen: After the conversation ends, the device displays the feedback sent from the server. It displays the evaluation: "It was a very clear closing offer, but there was a lack of specific pricing information."

[0945] User

[0946] Users operate this system to simulate interactions with actual customers. Furthermore, an emotion engine analyzes those emotions in real time.

[0947] 1. Scenario Selection: The user selects the scenario they want to practice from the scenario selection screen on their device. Select "Consultation on purchasing home appliances" to start the scenario.

[0948] 2. Virtual Conversation Progression: The user progresses the conversation by responding to messages from the customer displayed on the terminal. For example, the user might type "What product are you looking for?" and continue the conversation with the customer.

[0949] 3. Closing Proposal: The user makes a closing proposal based on the flow of the conversation. For example, they might suggest, "Our A Company refrigerator uses the latest energy-saving technology and can reduce your electricity bill in the long run."

[0950] 4. Emotion Monitoring: The user's facial expressions and tone of voice are analyzed in real time and sent to the server. For example, if the user shows an anxious expression, this is analyzed and influences the next scenario.

[0951] 5. Receiving Feedback: After the conversation ends, the user reviews the feedback displayed on their device and uses it to improve their next practice session. For example, they might receive feedback such as, "The closing offer was very clear, but there was a lack of specific pricing information."

[0952] Usage examples and prompt messages

[0953] Usage example

[0954] The user selects the "consultation on purchasing home appliances" scenario on their device, and the conversation begins.

[0955] The first message displayed is, "Hello, I'm thinking of buying some home appliances."

[0956] The user types, "What kind of product are you looking for?", and the conversation proceeds.

[0957] The server generates a customer response saying, "I'm considering buying a refrigerator. Could you tell me the difference between products from company A and company B?" and sends it to the terminal.

[0958] A user suggests, "Our A-brand refrigerator uses the latest energy-saving technology, which will save you money on electricity bills."

[0959] Based on the conversation log and sentiment analysis results, the server generates feedback stating, "This was a very clear closing offer, but it lacked specific pricing information," and displays it on the terminal.

[0960] Example of a prompt

[0961] "Next scenario: Inquiry about purchasing home appliances. Please generate an initial message for the customer."

[0962] "Analyze the user's input and generate the next customer's response."

[0963] "Generate feedback and evaluation for the user's closing suggestion."

[0964] This system allows new employees to efficiently improve their closing proposal skills in a practical environment. Furthermore, the emotion engine enables advanced practice that takes user emotions into account.

[0965] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0966] Step 1:

[0967] The server receives scenario selection information sent from the terminal. When the user selects "consultation on purchasing home appliances" on the terminal, the server receives that information and performs the appropriate database query. The input is the user's scenario selection information, and the output is the database query related to the scenario.

[0968] Step 2:

[0969] The server retrieves customer role information from the database based on the scenario selection. It uses SQL queries to retrieve customer attribute information and past sales history related to "consultation on purchasing home appliances." The input is a database query, and the output is specific information about the customer role.

[0970] Step 3:

[0971] The server generates a scenario using a template based on the acquired customer role information. For example, it creates a specific scenario such as "The customer is interested in refrigerators from companies A and B." The input is customer role information and a template, and the output is the specific scenario.

[0972] Step 4:

[0973] The server sends the generated scenario to the user's terminal and displays the initial message of the scenario. For example, it might display the message, "Hello, I'm thinking about buying some home appliances." The input is the generated scenario, and the output is the initial message displayed on the terminal.

[0974] Step 5:

[0975] The terminal accepts user input in real time. When a user types "What product are you looking for?", that information is sent to the server. The input is the user's message, and the output is the input data sent to the server.

[0976] Step 6:

[0977] The server analyzes user input and generates the next customer response based on that input. It uses a generative AI model to generate appropriate responses to user questions. For example, it might generate the response, "I'm looking at refrigerators. Could you tell me the differences between products from company A and company B?" The input is the user's input data, and the output is the generated customer response.

[0978] Step 7:

[0979] The server sends the generated customer response to the user's terminal for display. The terminal displays the content and prompts the user for the next action. The input is the generated customer response, and the output is the message displayed on the terminal.

[0980] Step 8:

[0981] The server saves logs of all virtual conversations. Each step of the conversation is saved chronologically in the database for later reference. The input is the conversation log data, and the output is the saved conversation log.

[0982] Step 9:

[0983] The server generates an evaluation and feedback on the user's closing proposal based on the conversation log. It analyzes the log and provides feedback such as, "This was a very clear closing proposal, but it lacked specific pricing." The input is the conversation log data, and the output is the feedback content.

[0984] Step 10:

[0985] The server sends the generated feedback to the user's terminal, which then displays the feedback. The input is the generated feedback, and the output is the evaluation and suggestions displayed on the terminal.

[0986] Step 11:

[0987] The device uses a camera and microphone to analyze the user's facial expressions and voice tone. It captures the user's face and voice in real time and sends it to the server. The input is data obtained from the camera and microphone, and the output is the analyzed data sent to the server.

[0988] Step 12:

[0989] The server uses emotion analysis tools to analyze the user's facial expressions and tone of voice, and recognizes their emotions. For example, if it detects an anxious facial expression or tone of voice, it makes positive changes to the next scenario based on the analysis results. The input is the analysis data sent from the terminal, and the output is the analysis results and the modified scenario.

[0990] (Application Example 2)

[0991] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0992] Traditional customer service training systems have made it difficult for new sales staff to effectively learn how to respond to real-world customer service scenarios. Furthermore, it has been challenging to conduct more realistic training by analyzing user emotions in real time and providing feedback based on that analysis. Therefore, there is a need for a system that can quickly and reliably improve the customer service skills of new sales staff.

[0993] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0994] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, an emotion recognition means that analyzes emotions in real time using a camera and a microphone, a means that transmits information including the analyzed emotion data to the server, and a means that outputs the customer role's response as voice by speech synthesis. This makes it possible for new sales staff to practice customer service skills in a realistic manner and to effectively improve their skills through feedback that includes real-time emotion analysis.

[0995] A "scenario selection method" is a means for a user to select a specific scenario to use in training.

[0996] The "customer role information acquisition method" is a means of acquiring relevant customer role information from a database based on the selected scenario.

[0997] A "scenario generation method" is a method for generating a scenario from a template based on acquired customer role information.

[0998] "User input receiving means" refers to means for receiving user input.

[0999] "Customer role response generation means" refers to means for generating the next response of the customer role based on user input.

[1000] A "conversation log saving method" is a means of saving logs of virtual conversations between a user and a customer.

[1001] A "feedback generation method" is a means of generating evaluations and feedback on a user's closing suggestion based on conversation logs.

[1002] "Evaluation display means" refers to a means of sending the generated feedback to a terminal and displaying it to the user.

[1003] "Emotion recognition means" refers to a method of analyzing a user's emotions in real time using a camera and microphone.

[1004] "Means for transmitting information including analyzed emotion data to a server" refers to means for transmitting emotion data analyzed by emotion recognition means to a server.

[1005] "Means for outputting customer responses as audio using speech synthesis" refers to means for outputting the generated customer responses as audio.

[1006] The system for realizing this invention consists of three components: a server, a terminal, and a user. The functions and roles of each component are described below.

[1007] server

[1008] The server plays a central role in generating and managing scenarios, and receiving and analyzing user input. The server includes the following:

[1009] 1. Scenario selection method:

[1010] When a user selects a specific scenario to use for training, the scenario selection mechanism receives that information.

[1011] 2. Means of obtaining customer role information:

[1012] Based on the selected scenario, relevant customer information is retrieved from the database. For example, in a scenario involving a consultation about purchasing home appliances, past purchase history and customer attribute information are retrieved.

[1013] 3. Scenario generation method:

[1014] Based on the acquired customer information, a scenario is generated from a template. This scenario includes instructions on how the user should respond.

[1015] 4. User input receiving means:

[1016] It receives user input in real time. For example, it receives information such as "What kind of product are you looking for?" entered by the user.

[1017] 5. Means for generating customer role responses:

[1018] Based on user input, the system generates the next response from the customer. For example, a response like, "I'm looking to buy a refrigerator. Could you tell me the difference between products from company A and company B?"

[1019] 6. Means of saving conversation logs:

[1020] The system saves a chronological log of virtual conversations between the user and the customer. This serves as reference information when generating feedback later.

[1021] 7. Feedback generation means:

[1022] Based on saved conversation logs, the system generates evaluations and feedback on the user's closing proposal. For example, it might provide feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1023] 8. Means of displaying evaluations:

[1024] The feedback generated by the feedback generation mechanism is sent to the terminal and displayed to the user.

[1025] 9. Emotion recognition means:

[1026] The system uses a camera and microphone to analyze the user's facial expressions and voice tone in real time. For example, if the user shows an expression of dissatisfaction, the system analyzes that emotional data.

[1027] 10. Means for transmitting information, including analyzed sentiment data, to a server:

[1028] The emotion data analyzed by the emotion recognition system is sent to the server.

[1029] 11. Means for outputting the customer's response as audio using speech synthesis:

[1030] The generated customer responses are output as audio. This allows users to receive training in a more realistic way.

[1031] terminal

[1032] The terminal provides an interface for the user to communicate with the server and engage in virtual conversations. It includes the following functions:

[1033] 1. Scenario selection screen:

[1034] The user is prompted to select a scenario on the initial screen.

[1035] 2. User input screen:

[1036] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[1037] 3. Response display screen:

[1038] Displays the customer's response to the user's input.

[1039] 4. Emotion Recognition Interface:

[1040] The device uses cameras and microphones to analyze the user's facial expressions and voice tone.

[1041] 5. Feedback screen:

[1042] After the conversation ends, the feedback sent from the server will be displayed.

[1043] User

[1044] Users operate this system to simulate interactions with actual customers.

[1045] 1. Scenario Selection:

[1046] Select the scenario you want to practice on the device's scenario selection screen. For example, select "Consultation on purchasing home appliances."

[1047] 2. The progress of the virtual conversation:

[1048] Respond to messages from the customer displayed on the terminal and advance the conversation.

[1049] 3. Closing proposal:

[1050] Based on the flow of the conversation, make a closing proposal. For example, you might suggest, "Our A-brand refrigerators use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[1051] 4. Emotional monitoring:

[1052] During the conversation, the user's facial expressions and voice tone are analyzed in real time, and that data is sent to the server.

[1053] 5. Feedback received:

[1054] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[1055] Specific example:

[1056] Examples of prompts to input into a generative AI model include the following:

[1057] Please select the "Consultation on purchasing home appliances" scenario. The initial message from the customer is "Hello, I'm thinking about purchasing home appliances." Next, respond with "What kind of products are you looking for?" If the user's expression seems anxious, make the customer's response more positive. Finally, make a closing suggestion by saying, "Here are some products I recommend."

[1058] In this way, by using this system, new sales staff can efficiently improve their customer service skills in a practical environment.

[1059] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1060] Step 1:

[1061] Scenario Selection

[1062] The server receives scenario information specified by the user via a scenario selection mechanism. The user selects the scenario they want to practice (for example, "consultation on purchasing home appliances") on the terminal's scenario selection screen. The input is scenario information, and the output is the system settings based on the selected scenario.

[1063] Step 2:

[1064] Acquisition of customer role information

[1065] The server uses a customer role information acquisition mechanism to retrieve relevant customer role information from the database based on the selected scenario. For example, past purchase history and attribute information of a customer considering purchasing home appliances. The input is a request for customer role information based on the scenario, and the output is the retrieved customer role information.

[1066] Step 3:

[1067] Scenario generation

[1068] The server generates a scenario from a template based on customer role information obtained via a scenario generation mechanism. For example, it might generate a scenario that includes the initial message, "Hello, I'm thinking of buying some home appliances." The input is customer role information, and the output is the generated scenario.

[1069] Step 4:

[1070] Receiving user input

[1071] The terminal displays customer-facing messages based on a scenario and accepts user input in real time. For example, the user might input, "What product are you looking for?" The input is the user's response, and the output is a record of that response.

[1072] Step 5:

[1073] Customer response generation

[1074] The server uses a customer response generation mechanism to analyze user input and generate the next customer response based on that input. For example, it might generate a response like, "I'm looking at refrigerators. Could you tell me the difference between products from company A and company B?" The input is the user's response, and the output is the generated customer response.

[1075] Step 6:

[1076] Saving conversation logs

[1077] The server uses a conversation log storage mechanism to save logs of virtual conversations between the user and the customer. The saved content includes the user's statements, the customer's generated responses, and timestamps. The input is each sequence of the conversation, and the output is the saved log.

[1078] Step 7:

[1079] Emotion analysis and transmission

[1080] The device uses a camera and microphone to capture the user's facial expressions and voice tone in real time, and analyzes them using emotion recognition technology. The analyzed emotion data is sent to a server. The input is the user's facial expressions and voice, and the output is the analyzed emotion data.

[1081] Step 8:

[1082] Audio output of the customer's response

[1083] The server outputs the customer's responses, generated using speech synthesis, as audio. This allows the user to train in more realistic scenarios. The input is the generated customer's responses, and the output is the audio response.

[1084] Step 9:

[1085] Generating and displaying feedback

[1086] The server generates an evaluation and feedback on the user's closing proposal using a feedback generation mechanism, based on the conversation log and analyzed sentiment data. This is then sent to the terminal using an evaluation display mechanism and displayed to the user. For example, feedback such as "This was a very clear closing proposal, but it lacked specific pricing" might be displayed. The input is the conversation log and sentiment data, and the output is the generated feedback.

[1087] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1088] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1089] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1090] [Third Embodiment]

[1091] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1092] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1093] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1094] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1095] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1096] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1097] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1098] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1099] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1100] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1101] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1102] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1103] The system of this invention is designed to provide a practice environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users (new employees), and its specific operation is described below.

[1104] server

[1105] The server plays a central role in generating scenarios and customer responses.

[1106] 1. Scenario selection method:

[1107] The server receives the scenario selection information specified by the user.

[1108] Specific example: The user selects the scenario "consultation on purchasing home appliances."

[1109] 2. Means of obtaining customer role information:

[1110] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[1111] Specific example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[1112] 3. Scenario generation method:

[1113] Based on the acquired customer information, a scenario is generated from a pre-prepared template.

[1114] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[1115] 4. User input receiving means:

[1116] The server receives user input from the terminal.

[1117] Specific example: Receive information entered by the user, such as "What kind of product are you looking for?".

[1118] 5. Means for generating customer role responses:

[1119] The server analyzes the received user input and generates the next response from the customer based on that input.

[1120] Specific example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[1121] 6. Means of saving conversation logs:

[1122] The server saves logs of virtual conversations between users and customers.

[1123] Specific example: Save each exchange in a conversation in chronological order.

[1124] 7. Feedback generation means:

[1125] Based on the conversation log, it generates an evaluation and feedback on the user's closing suggestion.

[1126] Specific example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1127] 8. Means of displaying evaluations:

[1128] The server sends the generated feedback to the terminal and displays it to the user.

[1129] Specific example: Display the feedback content on the user's device.

[1130] terminal

[1131] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[1132] 1. Scenario selection screen:

[1133] The user is prompted to select a scenario on the initial screen.

[1134] Specific example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[1135] 2. User input screen:

[1136] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[1137] Specific example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[1138] 3. Response display screen:

[1139] Displays the customer's response to the user's input.

[1140] Specific example: The server displays the response, "I am considering purchasing a refrigerator. Please tell me the differences between products from company A and company B."

[1141] 4. Feedback screen:

[1142] After the conversation ends, the feedback sent from the server will be displayed.

[1143] Specific example: Display a review stating, "The closing proposal was very clear, but it lacked a specific price."

[1144] User

[1145] Users operate this system to simulate interactions with actual customers.

[1146] 1. Scenario Selection:

[1147] The user selects the scenario they want to practice from the scenario selection screen on their device.

[1148] Example: Select "Consultation on purchasing home appliances".

[1149] 2. The progress of the virtual conversation:

[1150] The user progresses the conversation by responding to messages from a customer displayed on their device.

[1151] Example: Continue the conversation with the customer by typing, "What kind of product are you looking for?"

[1152] 3. Closing proposal:

[1153] Based on the flow of the conversation, the user makes a closing proposal.

[1154] Specific example: "Our refrigerators from Company A use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[1155] 4. Feedback received:

[1156] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[1157] Specific example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[1158] In this way, by using this system, new employees can efficiently improve their closing proposal skills in a practical environment.

[1159] The following describes the processing flow.

[1160] Step 1:

[1161] The user opens the scenario selection screen using their device and specifies the scenario they want to practice. The scenario selection information is sent to the server.

[1162] Step 2:

[1163] The server analyzes the scenario selection information received from the user and retrieves customer role information related to the selected scenario from the database.

[1164] Step 3:

[1165] Based on the customer information acquired by the server, a scenario is generated from a pre-prepared template. The generated scenario data is then sent to the terminal.

[1166] Step 4:

[1167] Based on the scenario data received from the server, the terminal displays an initial message to the user, who is playing the role of a customer. The user then enters a response to the initial message.

[1168] Step 5:

[1169] User input is sent from the terminal to the server. The server analyzes the user input and generates the next customer response based on its content.

[1170] Step 6:

[1171] The server sends the next response generated by the customer role to the terminal. The terminal displays this to the user. The user then enters their response again.

[1172] Step 7:

[1173] Steps 5 and 6 are repeated, and a virtual conversation between the user and the customer progresses. The user makes closing suggestions based on the flow of the conversation and continues to input.

[1174] Step 8:

[1175] Once the conversation ends, the terminal sends all conversation logs to the server. The server saves the conversation logs and begins analysis.

[1176] Step 9:

[1177] The server generates an evaluation and feedback on the user's closing suggestion based on the conversation log. The generated feedback is then sent to the terminal.

[1178] Step 10:

[1179] The device displays feedback received from the server to the user. The user reviews the feedback and uses it to improve their next practice session.

[1180] (Example 1)

[1181] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1182] Providing an effective training environment for new employees to practically improve their closing proposal skills is challenging. Traditional training lacks real-time feedback and practice using concrete scenarios, hindering the development of skills relevant to actual work. To address this challenge, a system is needed that provides real-time quantitative and qualitative feedback through virtual customer interactions.

[1183] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1184] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a user input means via a terminal, a scenario display means via a terminal, a customer role response display means via a terminal, a feedback display means via a terminal, a means for acquiring customer role information from a database via the server, a scenario generation means via the server, a user input analysis means via the server, a customer role message display means via a terminal, and a means for generating and displaying evaluations and feedback based on the conversation log. This enables the user to effectively practice dialogue skills and closing proposal skills through real-time virtual conversations based on scenarios and to receive detailed feedback.

[1185] A "scenario selection method" is a function that provides an interface for users to select the scenario they want to practice.

[1186] The "customer role information acquisition method" is a function that retrieves information about customer roles related to a scenario from a database.

[1187] The "scenario generation method" is a function that generates a scenario using a template based on the acquired customer role information.

[1188] A "user input receiving means" is a function that receives input from the user and analyzes it.

[1189] The "customer role response generation means" is a function that generates the next response of the customer role based on user input.

[1190] The "conversation log saving method" is a function that saves a chronological log of a virtual conversation between the user and the customer.

[1191] The "feedback generation method" is a function that generates evaluations and feedback on the user's closing suggestion based on the saved conversation log.

[1192] "Evaluation display means" refers to a function that displays the generated feedback to the user.

[1193] "User input means via terminal" refers to a function that accepts user input in real time.

[1194] "Scenario display means via terminal" refers to a function that displays the generated scenario to the user.

[1195] "Means for displaying customer responses via a terminal" refers to a function that displays the customer's response to user input.

[1196] "Means of displaying feedback via the terminal" refers to a function that displays the generated feedback to the user after the conversation has ended.

[1197] "Means of retrieving customer role information from the database by the server" refers to the function of retrieving customer role information from the database.

[1198] "Server-based scenario generation method" refers to a function that generates scenarios using templates based on customer information.

[1199] "Server-based user input analysis means" refers to a function that analyzes user input and generates the next customer response based on that analysis.

[1200] "Means for displaying customer-role messages via a terminal" refers to a function that displays messages from the customer role to the user.

[1201] "Means for generating and displaying evaluations and feedback based on conversation logs" refers to a function that generates evaluations and feedback on a user's closing suggestion based on saved conversation logs and displays them to the user.

[1202] The present invention provides a training environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users, and each component works in cooperation with the others.

[1203] System Configuration

[1204] server

[1205] The server plays a central role in the system, providing the means to generate scenarios and customer responses. The server has the following functions:

[1206] 1. Scenario Selection Method

[1207] The server receives scenario selection information specified by the user.

[1208] Example: The user selects the scenario "Consultation on purchasing home appliances".

[1209] 2. Means for obtaining customer role information

[1210] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[1211] Example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[1212] 3. Scenario generation means

[1213] The server generates a scenario from a template based on the customer role information it has acquired.

[1214] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[1215] 4. User input receiving means

[1216] The server receives user input from the terminal.

[1217] Example: Receive information entered by the user, such as "What kind of product are you looking for?".

[1218] 5. Customer response generation means

[1219] The server analyzes the user's input and generates the next customer's response based on that input.

[1220] Example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[1221] 6. Means of saving conversation logs

[1222] The server saves logs of virtual conversations between users and customers.

[1223] Example: Save all conversations in chronological order.

[1224] 7. Feedback generation means

[1225] The server generates an evaluation and feedback on the user's closing suggestion based on the saved conversation logs.

[1226] Example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1227] 8. Evaluation display means

[1228] The server sends the generated feedback to the terminal and displays it to the user.

[1229] Example: Display the feedback content on the user's device.

[1230] terminal

[1231] The terminal provides an interface for users to engage in virtual conversations through communication with the server. The terminal's functions are as follows:

[1232] 1. Scenario Selection Screen

[1233] The user is prompted to select a scenario on the initial screen.

[1234] Example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[1235] 2. User input screen

[1236] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[1237] Example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[1238] 3. Response display screen

[1239] Displays the customer's response to the user's input.

[1240] Example: The server will display the response, "I am considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[1241] 4. Feedback screen

[1242] After the conversation ends, the feedback sent from the server will be displayed.

[1243] Example: Display a review such as, "It was a very clear closing proposal, but it lacked a specific price."

[1244] User

[1245] The user operates this system to simulate interactions with actual customers. Follow these steps:

[1246] 1. Scenario Selection

[1247] The user selects the scenario they want to practice from the scenario selection screen on their device.

[1248] Example: Select "Consultation on purchasing home appliances".

[1249] 2. Progress of the virtual conversation

[1250] The user progresses the conversation by responding to messages from a customer displayed on their device.

[1251] Example: Continue the conversation with the customer by typing "What kind of product are you looking for?"

[1252] 3. Closed proposal

[1253] Based on the flow of the conversation, the user makes a closing proposal.

[1254] Example: "Our A Company refrigerators use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[1255] 4. Receiving Feedback

[1256] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[1257] Example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[1258] Example of a prompt

[1259] "Analyze the following user input and generate an appropriate response to the virtual customer's question: 'What products are you looking for?'"

[1260] In this way, by using this system, new employees can efficiently improve their closing proposal skills in a practical environment.

[1261] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1262] Step 1: Scenario Selection

[1263] The server receives scenario selection information sent from the terminal. Based on this input information, the server recognizes the selected scenario and prepares for the next processing. For example, if a user selects the scenario "consultation on purchasing home appliances," that information is sent to the server.

[1264] Step 2: Obtain customer role information

[1265] The server uses the scenario selection information to retrieve relevant customer role information from the database. This process involves executing database queries and extracting the necessary data. The retrieved data includes customer attribute information and past sales history. For example, the server retrieves attribute information and past sales history for a "customer considering purchasing home appliances."

[1266] Step 3: Scenario Generation

[1267] The server generates a scenario using a template based on the customer role information it has acquired. In this process, data is applied to the template to generate a specific conversation scenario. The generated scenario includes an initial message and questions. For example, the server generates a scenario based on "consultation about purchasing home appliances" and creates an initial message such as "Hello, I'm thinking about purchasing home appliances."

[1268] Step 4: Display the scenario

[1269] The terminal displays the scenario sent from the server to the user. In this process, the terminal converts the received data into an appropriate layout for display on the screen. It receives scenario data from the server as input and displays it to the user as output. For example, the terminal displays the message "Hello, I'm thinking about buying home appliances" to the user.

[1270] Step 5: User response input

[1271] The user enters a response to a customer-facing message displayed on the terminal. The entered data is sent from the terminal to the server. For example, the user might enter "What product are you looking for?"

[1272] Step 6: Receiving and analyzing user input

[1273] The server analyzes user input received from the terminal. This process uses natural language processing techniques to understand the user's intent and generate the next customer response. It receives the user's message as input, analyzes it, and then generates the next response data. For example, the server analyzes the input "What kind of product are you looking for?" and generates the response "I'm considering a refrigerator. Could you tell me the difference between products from company A and company B?"

[1274] Step 7: Customer's reaction

[1275] The terminal displays customer responses generated by the server to the user. In this process, it receives response data from the server and displays it appropriately on the screen. It receives response data as input and displays it to the user as output. For example, the terminal displays, "I'm considering buying a refrigerator. Could you tell me the differences between products from company A and company B?"

[1276] Step 8: Saving the conversation log

[1277] The server saves logs of virtual conversations between the user and the customer. This process stores all conversational exchanges chronologically in a database. It takes conversational data as input and saves it to the database as output. For example, the server saves each individual exchange in a conversation as a log.

[1278] Step 9: Generating Feedback

[1279] The server generates an evaluation and feedback on the user's closing proposal based on the saved conversation logs. This process analyzes the conversation logs and creates feedback based on evaluation criteria. It takes conversation logs as input and generates feedback data as output. For example, the server might generate feedback such as, "This was a very clear closing proposal, but it lacked a specific price offer."

[1280] Step 10: Displaying Feedback

[1281] The terminal displays feedback sent from the server to the user. This process converts the feedback data into an appropriate layout for display on the screen. It receives feedback data as input and displays it to the user as output. For example, the terminal displays the feedback, "It was a very clear closing offer, but it lacked specific pricing information."

[1282] (Application Example 1)

[1283] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1284] For new employees to efficiently improve skills such as presentations and content proposals, practice in a realistic environment is necessary. However, many companies have limited employees and resources for such practice, making it difficult to provide effective training. Furthermore, there are challenges such as limited opportunities for individual feedback and difficulty in identifying areas for improvement through self-assessment.

[1285] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1286] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a response training means including scenario selection information, and a prompt sentence generation means. This allows the user to practice in real time through interaction with a virtual customer and receive individualized feedback.

[1287] "Scenario selection method" refers to the means by which users select a scenario for practice.

[1288] "Method for obtaining customer role information" refers to a means for obtaining information about customer roles related to the selected scenario from a database.

[1289] A "scenario generation method" is a means of generating a scenario from a pre-prepared template based on acquired customer role information.

[1290] "User input receiving means" refers to means for receiving user input.

[1291] "Customer role response generation means" refers to means for generating the next response of the customer role based on the received user input.

[1292] A "conversation log saving method" is a means for saving logs of virtual conversations between a user and a customer.

[1293] A "feedback generation method" is a means of generating evaluations and feedback on a user's closing suggestion based on conversation logs.

[1294] "Evaluation display means" refers to a means for displaying the generated feedback to the user.

[1295] A "response training method including scenario selection information" is a means for training responses based on scenario selection information specified by the user.

[1296] A "prompt message generation means" is a means for generating prompt messages that users will use.

[1297] The present invention is a system for efficiently improving the presentation skills and content proposal abilities of new employees. This system includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a response training means including scenario selection information, and a prompt sentence generation means.

[1298] server

[1299] Scenario selection method

[1300] The server receives information from the user about the scenario they want to practice. For example, if the user selects "New Product Presentation Scenario," the server processes this information.

[1301] Customer information acquisition method

[1302] The server retrieves customer role information from the database based on the selected scenario. This information includes customer profiles and past conversation history.

[1303] Scenario generation method

[1304] Based on the customer information obtained, the server generates an appropriate scenario from pre-prepared templates. For example, it might generate a "presentation scenario emphasizing the advantages of a new product."

[1305] User input receiving means

[1306] The server receives user input (for example, a question like, "How is the new product better?") in real time.

[1307] Customer response generation means

[1308] The server analyzes user input and generates the next response from the customer role based on that input. For example, in response to user input, it might generate a response such as, "The new product incorporates the latest technology and is very easy to use."

[1309] Conversation log saving method

[1310] The server stores a chronological log of the conversation between the user and the customer. This log is used later to generate feedback.

[1311] Feedback generation means

[1312] Based on the conversation log, the server generates user closing suggestions and an overall evaluation and feedback on the presentation. For example, it might provide feedback such as, "The presentation was generally easy to understand, but it lacked detailed explanations of the specifications."

[1313] Evaluation display means

[1314] The server sends the generated feedback to the user's device, allowing the user to check their performance.

[1315] terminal

[1316] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[1317] Scenario selection screen

[1318] The initial screen allows the user to select a scenario. For example, a screen might be displayed where the user can select "New Product Presentation Scenario."

[1319] User input screen

[1320] It displays an initial message from a customer based on a scenario and accepts user input in real time. For example, it might display the message, "Hello, please give me a presentation on the new product," and then receive a response from the user.

[1321] Response display screen

[1322] It displays the customer's response to the user's input. For example, it might display the server's response: "The new product incorporates the latest technology and is very easy to use."

[1323] Feedback screen

[1324] After the conversation ends, the server displays the feedback it has received. For example, it might show an evaluation such as, "The presentation was generally easy to understand, but it lacked detailed explanations of the specifications."

[1325] User

[1326] Scenario Selection

[1327] The user selects the scenario they want to practice from the scenario selection screen on their device. For example, they might select "New Product Presentation Scenario."

[1328] Progress of the virtual conversation

[1329] The user progresses the conversation by responding to messages from a customer displayed on the device. For example, they might type, "How is the new product better?" to continue the conversation.

[1330] Closed proposal

[1331] Based on the flow of the conversation, the user makes a closing suggestion. For example, they might suggest, "This new product is reasonably priced, so please give it a try."

[1332] Feedback received

[1333] After the conversation ends, review the feedback displayed on the device and use it to improve your next practice session. For example, you might receive feedback such as, "The presentation was generally easy to understand, but there was a lack of detailed explanation of the specifications," and use that to identify areas for improvement.

[1334] Specific examples and prompt statements

[1335] This example illustrates how new content creators can learn to present new content. An example of a prompt is: "Our new content offers clear visuals and intuitive controls that will excite viewers. Is there anything specific you're concerned about?" In this way, newcomers can effectively improve their skills by utilizing scenarios provided by the server and real-time feedback.

[1336] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1337] Step 1:

[1338] The server receives scenario selection information from the user through a scenario selection mechanism. Specifically, the user selects the scenario they want to practice and sends that information to the server. The input is scenario selection information, and the output is the selected scenario information.

[1339] Step 2:

[1340] The server uses a customer role information acquisition mechanism to retrieve customer role information related to the selected scenario from the database. This includes data such as the customer role's profile and past conversation history. The input is the selected scenario information, and the output is the retrieved customer role information.

[1341] Step 3:

[1342] The server uses a scenario generation mechanism to generate a scenario based on the acquired customer role information. A specific scenario is created using a template. The input is customer role information and a template, and the output is the generated scenario.

[1343] Step 4:

[1344] The server receives user input through a user input receiving mechanism. The user inputs questions and suggestions for the customer role based on a scenario. The input is direct information from the user, and the output is that same input information.

[1345] Step 5:

[1346] The server uses a customer role response generation mechanism to analyze the received user input and generates the next customer role response using a generation AI model. The input is the user's input information, and the output is the generated customer role response.

[1347] Step 6:

[1348] The server uses a conversation log storage mechanism to save logs of virtual conversations between the user and the customer. The input consists of the user's input information and the customer's responses, and the output is the saved conversation log.

[1349] Step 7:

[1350] The server uses a feedback generation mechanism to generate an evaluation and feedback on the user's closing suggestion based on the saved conversation log. The input is the saved conversation log, and the output is the generated feedback information.

[1351] Step 8:

[1352] The server displays the generated feedback to the user through the evaluation display means. The feedback content is displayed on the user's terminal. The input is the feedback information, and the output is the feedback displayed on the user's terminal.

[1353] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1354] The system of this invention is designed to provide a practice environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users (new employees), and its specific operation is described below.

[1355] server

[1356] The server plays a central role in generating scenarios and customer responses. It also uses an emotion engine to analyze user emotions and reflect them in the system's responses.

[1357] 1. Scenario selection method:

[1358] The server receives the scenario selection information specified by the user.

[1359] Specific example: The user selects the scenario "consultation on purchasing home appliances."

[1360] 2. Means of obtaining customer role information:

[1361] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[1362] Specific example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[1363] 3. Scenario generation method:

[1364] Based on the acquired customer information, a scenario is generated from a pre-prepared template.

[1365] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[1366] 4. User input receiving means:

[1367] The server receives user input from the terminal.

[1368] Specific example: Receive information entered by the user, such as "What kind of product are you looking for?".

[1369] 5. Means for generating customer role responses:

[1370] The server analyzes the received user input and generates the next response from the customer based on that input.

[1371] Specific example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[1372] 6. Means of saving conversation logs:

[1373] The server saves logs of virtual conversations between users and customers.

[1374] Specific example: Save each exchange in a conversation in chronological order.

[1375] 7. Feedback generation means:

[1376] Based on the conversation log, it generates an evaluation and feedback on the user's closing suggestion.

[1377] Specific example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1378] 8. Means of displaying evaluations:

[1379] The server sends the generated feedback to the terminal and displays it to the user.

[1380] Specific example: Display the feedback content on the user's device.

[1381] 9. Emotional Engine:

[1382] The server uses an emotion engine to analyze the user's facial expressions and tone of voice to recognize their emotions.

[1383] Specific example: If a user shows an expression of dissatisfaction, analyze that emotional data and generate a scenario that includes positive information.

[1384] terminal

[1385] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[1386] 1. Scenario selection screen:

[1387] The user is prompted to select a scenario on the initial screen.

[1388] Specific example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[1389] 2. User input screen:

[1390] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[1391] Specific example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[1392] 3. Response display screen:

[1393] Displays the customer's response to the user's input.

[1394] Specific example: The server displays the response, "I am considering purchasing a refrigerator. Please tell me the differences between products from company A and company B."

[1395] 4. Emotion Recognition Interface:

[1396] The device uses cameras and microphones to analyze the user's facial expressions and voice tone.

[1397] Specific example: Sending the tone of voice and facial expressions of a user to the server in real time when they speak.

[1398] 5. Feedback screen:

[1399] After the conversation ends, the feedback sent from the server will be displayed.

[1400] Specific example: Display a review stating, "The closing proposal was very clear, but it lacked a specific price."

[1401] User

[1402] Users operate this system to simulate interactions with actual customers. Furthermore, an emotion engine analyzes those emotions in real time.

[1403] 1. Scenario Selection:

[1404] The user selects the scenario they want to practice from the scenario selection screen on their device.

[1405] Example: Select "Consultation on purchasing home appliances".

[1406] 2. The progress of the virtual conversation:

[1407] The user progresses the conversation by responding to messages from a customer displayed on their device.

[1408] Example: Continue the conversation with the customer by typing, "What kind of product are you looking for?"

[1409] 3. Closing proposal:

[1410] Based on the flow of the conversation, the user makes a closing proposal.

[1411] Specific example: "Our refrigerators from Company A use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[1412] 4. Emotional monitoring:

[1413] During the conversation, the user's facial expressions and voice tone are analyzed in real time, and that data is sent to the server.

[1414] Specific example: If a user shows signs of anxiety, this is analyzed and influences the next scenario.

[1415] 5. Feedback received:

[1416] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[1417] Specific example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[1418] Thus, by using this system, new employees can not only efficiently improve their closing proposal skills in a practical environment, but also engage in more advanced practice that takes user emotions into consideration.

[1419] The following describes the processing flow.

[1420] Step 1:

[1421] The user opens the scenario selection screen using their device and specifies the scenario they want to practice. The scenario selection information is sent to the server.

[1422] Step 2:

[1423] The server analyzes the scenario selection information received from the user and retrieves customer role information related to the selected scenario from the database.

[1424] Step 3:

[1425] Based on the customer information acquired by the server, a scenario is generated from a pre-prepared template. The generated scenario data is then sent to the terminal.

[1426] Step 4:

[1427] Based on the scenario data received from the server, the terminal displays an initial message to the user, who is playing the role of a customer. The user then enters a response to the initial message.

[1428] Step 5:

[1429] User input is sent from the terminal to the server. The server analyzes the user input and generates the next customer response based on its content.

[1430] Step 6:

[1431] The server sends the next response generated by the customer role to the terminal. The terminal displays this to the user. The user then enters their response again.

[1432] Step 7:

[1433] The device uses its built-in camera and microphone to transmit the user's facial expressions and voice tone to the emotion engine in real time. The emotion engine analyzes this data to identify the user's emotional state.

[1434] Step 8:

[1435] The emotion engine recognizes the user's emotional state, which is then sent to the server. The server uses this information to adjust the customer's next response.

[1436] Step 9:

[1437] The server sends a customer's response, generated after considering emotional data, to the terminal. The terminal then displays this to the user.

[1438] Step 10:

[1439] The process from Step 5 to Step 9 is repeated, and a virtual conversation between the user and the customer progresses. The user makes closing suggestions based on the flow of the conversation and continues to input. In addition, the emotion engine monitors the user's emotional changes in real time.

[1440] Step 11:

[1441] Once the conversation ends, the terminal sends all conversation logs to the server. The server saves the conversation logs and begins analysis.

[1442] Step 12:

[1443] The server generates an evaluation and feedback on the user's closing suggestion based on the conversation log. The generated feedback is then sent to the terminal.

[1444] Step 13:

[1445] The device displays feedback received from the server to the user. The user reviews the feedback and uses it to improve their next practice session.

[1446] Through the above series of steps, users can improve their closing proposal skills through practical and emotionally responsive virtual conversations.

[1447] (Example 2)

[1448] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1449] Conventional systems for improving closing proposal skills for new employees suffer from monotonous customer responses, making it difficult to simulate the diverse reactions that occur in actual business negotiations. Furthermore, the lack of feedback that takes user emotions into account prevents effective practice that is relevant to real-world situations.

[1450] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, and an emotion analysis means. This enables the generation of diverse customer role responses that reflect the user's emotions, allowing practice in an environment close to an actual business negotiation.

[1451] "Scenario selection means" refers to a device or method that provides an interface for a user to select a practice scenario.

[1452] "Customer role information acquisition means" refers to a device or method for acquiring information about a virtual customer (customer role) from a database based on a selected scenario.

[1453] A "scenario generation means" is a device or method that generates a specific scenario by combining customer information with a pre-prepared template.

[1454] "User input receiving means" refers to a device or method for receiving content entered by a user from a terminal.

[1455] "Customer role response generation means" refers to a device or method that generates the next customer role response based on user input.

[1456] "Conversation log storage means" refers to a device or method for saving all logs of a virtual conversation.

[1457] "Feedback generation means" refers to a device or method that generates an evaluation and feedback on a user's closing suggestion based on a saved conversation log.

[1458] "Evaluation display means" refers to a device or method that displays the generated feedback on the user's terminal.

[1459] "Emotion analysis means" refers to a device or method for analyzing a user's facial expressions and tone of voice to recognize their emotions.

[1460] This invention is a system that provides a practice environment for effectively improving the closing proposal skills of new employees. The system consists of a server, terminals, and users, and their respective roles and operations are described below.

[1461] server

[1462] The server plays a central role in generating scenarios and customer responses to user input.

[1463] 1. Scenario Selection Method: The server receives scenario information specified by the user from the terminal. When the user selects "Consultation on purchasing home appliances," that information is sent to the server. The server then performs an appropriate database query based on this information.

[1464] 2. Means for obtaining customer role information: The server obtains customer role information from the database based on the selected scenario. For example, in the "consultation on purchasing home appliances" scenario, past sales history and customer attribute information are obtained using SQL queries.

[1465] 3. Scenario generation method: The server combines the acquired customer role information with templates to generate a specific scenario. For example, it creates a specific scenario such as "The customer is interested in refrigerators from companies A and B."

[1466] 4. User Input Reception Method: The server receives the information entered by the user from the terminal in real time. When the user enters "What product are you looking for?", that information is sent to the server and received by the server.

[1467] 5. Customer Role Response Generation Means: The server analyzes the received user input and generates the next customer role response based on its content. It uses a generation AI model to respond appropriately to the user's questions.

[1468] 6. Conversation log storage method: The server will save all logs of the virtual conversation. Each step of the conversation will be saved chronologically in the database for later reference.

[1469] 7. Feedback generation mechanism: The server generates an evaluation and feedback on the user's closing proposal based on the conversation log. It analyzes the log and provides feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1470] 8. Evaluation display means: The server sends the generated feedback to the user's terminal for display.

[1471] 9. Emotion Analysis Method: The server analyzes the user's facial expressions and tone of voice to recognize their emotions. It also analyzes data obtained from the camera and microphone and incorporates it into the scenario as needed.

[1472] terminal

[1473] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[1474] 1. Scenario Selection Screen: The device will prompt the user to select a scenario on the initial screen. It will display multiple scenarios, such as "Consultation on purchasing home appliances," and wait until the user makes a selection.

[1475] 2. User Input Screen: The terminal displays an initial message for the customer role based on a scenario and accepts user input in real time. For example, it might display the message, "Hello, I'm thinking of buying a home appliance."

[1476] 3. Response Display Screen: The terminal displays the customer's response to the user's input. It displays the server's response: "I'm considering buying a refrigerator. Please tell me the differences between Company A's and Company B's products."

[1477] 4. Emotion Recognition Interface: The device uses a camera and microphone to analyze the user's facial expressions and voice tone. It captures the user's face and voice in real time and sends it to the server.

[1478] 5. Feedback screen: After the conversation ends, the device displays the feedback sent from the server. It displays the evaluation: "It was a very clear closing offer, but there was a lack of specific pricing information."

[1479] User

[1480] Users operate this system to simulate interactions with actual customers. Furthermore, an emotion engine analyzes those emotions in real time.

[1481] 1. Scenario Selection: The user selects the scenario they want to practice from the scenario selection screen on their device. Select "Consultation on purchasing home appliances" to start the scenario.

[1482] 2. Virtual Conversation Progression: The user progresses the conversation by responding to messages from the customer displayed on the terminal. For example, the user might type "What product are you looking for?" and continue the conversation with the customer.

[1483] 3. Closing Proposal: The user makes a closing proposal based on the flow of the conversation. For example, they might suggest, "Our A Company refrigerator uses the latest energy-saving technology and can reduce your electricity bill in the long run."

[1484] 4. Emotion Monitoring: The user's facial expressions and tone of voice are analyzed in real time and sent to the server. For example, if the user shows an anxious expression, this is analyzed and influences the next scenario.

[1485] 5. Receiving Feedback: After the conversation ends, the user reviews the feedback displayed on their device and uses it to improve their next practice session. For example, they might receive feedback such as, "The closing offer was very clear, but there was a lack of specific pricing information."

[1486] Usage examples and prompt messages

[1487] Usage example

[1488] The user selects the "consultation on purchasing home appliances" scenario on their device, and the conversation begins.

[1489] The first message displayed is, "Hello, I'm thinking of buying some home appliances."

[1490] The user types, "What kind of product are you looking for?", and the conversation proceeds.

[1491] The server generates a customer response saying, "I'm considering buying a refrigerator. Could you tell me the difference between products from company A and company B?" and sends it to the terminal.

[1492] A user suggests, "Our A-brand refrigerator uses the latest energy-saving technology, which will save you money on electricity bills."

[1493] Based on the conversation log and sentiment analysis results, the server generates feedback stating, "This was a very clear closing offer, but it lacked specific pricing information," and displays it on the terminal.

[1494] Example of a prompt

[1495] "Next scenario: Inquiry about purchasing home appliances. Please generate an initial message for the customer."

[1496] "Analyze the user's input and generate the next customer's response."

[1497] "Generate feedback and evaluation for the user's closing suggestion."

[1498] This system allows new employees to efficiently improve their closing proposal skills in a practical environment. Furthermore, the emotion engine enables advanced practice that takes user emotions into account.

[1499] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1500] Step 1:

[1501] The server receives scenario selection information sent from the terminal. When the user selects "consultation on purchasing home appliances" on the terminal, the server receives that information and performs the appropriate database query. The input is the user's scenario selection information, and the output is the database query related to the scenario.

[1502] Step 2:

[1503] The server retrieves customer role information from the database based on the scenario selection. It uses SQL queries to retrieve customer attribute information and past sales history related to "consultation on purchasing home appliances." The input is a database query, and the output is specific information about the customer role.

[1504] Step 3:

[1505] The server generates a scenario using a template based on the acquired customer role information. For example, it creates a specific scenario such as "The customer is interested in refrigerators from companies A and B." The input is customer role information and a template, and the output is the specific scenario.

[1506] Step 4:

[1507] The server sends the generated scenario to the user's terminal and displays the initial message of the scenario. For example, it might display the message, "Hello, I'm thinking about buying some home appliances." The input is the generated scenario, and the output is the initial message displayed on the terminal.

[1508] Step 5:

[1509] The terminal accepts user input in real time. When a user types "What product are you looking for?", that information is sent to the server. The input is the user's message, and the output is the input data sent to the server.

[1510] Step 6:

[1511] The server analyzes user input and generates the next customer response based on that input. It uses a generative AI model to generate appropriate responses to user questions. For example, it might generate the response, "I'm looking at refrigerators. Could you tell me the differences between products from company A and company B?" The input is the user's input data, and the output is the generated customer response.

[1512] Step 7:

[1513] The server sends the generated customer response to the user's terminal for display. The terminal displays the content and prompts the user for the next action. The input is the generated customer response, and the output is the message displayed on the terminal.

[1514] Step 8:

[1515] The server saves logs of all virtual conversations. Each step of the conversation is saved chronologically in the database for later reference. The input is the conversation log data, and the output is the saved conversation log.

[1516] Step 9:

[1517] The server generates an evaluation and feedback on the user's closing proposal based on the conversation log. It analyzes the log and provides feedback such as, "This was a very clear closing proposal, but it lacked specific pricing." The input is the conversation log data, and the output is the feedback content.

[1518] Step 10:

[1519] The server sends the generated feedback to the user's terminal, which then displays the feedback. The input is the generated feedback, and the output is the evaluation and suggestions displayed on the terminal.

[1520] Step 11:

[1521] The device uses a camera and microphone to analyze the user's facial expressions and voice tone. It captures the user's face and voice in real time and sends it to the server. The input is data obtained from the camera and microphone, and the output is the analyzed data sent to the server.

[1522] Step 12:

[1523] The server uses emotion analysis tools to analyze the user's facial expressions and tone of voice, and recognizes their emotions. For example, if it detects an anxious facial expression or tone of voice, it makes positive changes to the next scenario based on the analysis results. The input is the analysis data sent from the terminal, and the output is the analysis results and the modified scenario.

[1524] (Application Example 2)

[1525] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1526] Traditional customer service training systems have made it difficult for new sales staff to effectively learn how to respond to real-world customer service scenarios. Furthermore, it has been challenging to conduct more realistic training by analyzing user emotions in real time and providing feedback based on that analysis. Therefore, there is a need for a system that can quickly and reliably improve the customer service skills of new sales staff.

[1527] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1528] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, an emotion recognition means that analyzes emotions in real time using a camera and a microphone, a means that transmits information including the analyzed emotion data to the server, and a means that outputs the customer role's response as voice by speech synthesis. This makes it possible for new sales staff to practice customer service skills in a realistic manner and to effectively improve their skills through feedback that includes real-time emotion analysis.

[1529] A "scenario selection method" is a means for a user to select a specific scenario to use in training.

[1530] The "customer role information acquisition method" is a means of acquiring relevant customer role information from a database based on the selected scenario.

[1531] A "scenario generation method" is a method for generating a scenario from a template based on acquired customer role information.

[1532] "User input receiving means" refers to means for receiving user input.

[1533] "Customer role response generation means" refers to means for generating the next response of the customer role based on user input.

[1534] A "conversation log saving method" is a means of saving logs of virtual conversations between a user and a customer.

[1535] A "feedback generation method" is a means of generating evaluations and feedback on a user's closing suggestion based on conversation logs.

[1536] "Evaluation display means" refers to a means of sending the generated feedback to a terminal and displaying it to the user.

[1537] "Emotion recognition means" refers to a method of analyzing a user's emotions in real time using a camera and microphone.

[1538] "Means for transmitting information including analyzed emotion data to a server" refers to means for transmitting emotion data analyzed by emotion recognition means to a server.

[1539] "Means for outputting customer responses as audio using speech synthesis" refers to means for outputting the generated customer responses as audio.

[1540] The system for realizing this invention consists of three components: a server, a terminal, and a user. The functions and roles of each component are described below.

[1541] server

[1542] The server plays a central role in generating and managing scenarios, and receiving and analyzing user input. The server includes the following:

[1543] 1. Scenario selection method:

[1544] When a user selects a specific scenario to use for training, the scenario selection mechanism receives that information.

[1545] 2. Means of obtaining customer role information:

[1546] Based on the selected scenario, relevant customer information is retrieved from the database. For example, in a scenario involving a consultation about purchasing home appliances, past purchase history and customer attribute information are retrieved.

[1547] 3. Scenario generation method:

[1548] Based on the acquired customer information, a scenario is generated from a template. This scenario includes instructions on how the user should respond.

[1549] 4. User input receiving means:

[1550] It receives user input in real time. For example, it receives information such as "What kind of product are you looking for?" entered by the user.

[1551] 5. Means for generating customer role responses:

[1552] Based on user input, the system generates the next response from the customer. For example, a response like, "I'm looking to buy a refrigerator. Could you tell me the difference between products from company A and company B?"

[1553] 6. Means of saving conversation logs:

[1554] The system saves a chronological log of virtual conversations between the user and the customer. This serves as reference information when generating feedback later.

[1555] 7. Feedback generation means:

[1556] Based on saved conversation logs, the system generates evaluations and feedback on the user's closing proposal. For example, it might provide feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1557] 8. Means of displaying evaluations:

[1558] The feedback generated by the feedback generation mechanism is sent to the terminal and displayed to the user.

[1559] 9. Emotion recognition means:

[1560] The system uses a camera and microphone to analyze the user's facial expressions and voice tone in real time. For example, if the user shows an expression of dissatisfaction, the system analyzes that emotional data.

[1561] 10. Means for transmitting information, including analyzed sentiment data, to a server:

[1562] The emotion data analyzed by the emotion recognition system is sent to the server.

[1563] 11. Means for outputting the customer's response as audio using speech synthesis:

[1564] The generated customer responses are output as audio. This allows users to receive training in a more realistic way.

[1565] terminal

[1566] The terminal provides an interface for the user to communicate with the server and engage in virtual conversations. It includes the following functions:

[1567] 1. Scenario selection screen:

[1568] The user is prompted to select a scenario on the initial screen.

[1569] 2. User input screen:

[1570] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[1571] 3. Response display screen:

[1572] Displays the customer's response to the user's input.

[1573] 4. Emotion Recognition Interface:

[1574] The device uses cameras and microphones to analyze the user's facial expressions and voice tone.

[1575] 5. Feedback screen:

[1576] After the conversation ends, the feedback sent from the server will be displayed.

[1577] User

[1578] Users operate this system to simulate interactions with actual customers.

[1579] 1. Scenario Selection:

[1580] Select the scenario you want to practice on the device's scenario selection screen. For example, select "Consultation on purchasing home appliances."

[1581] 2. The progress of the virtual conversation:

[1582] Respond to messages from the customer displayed on the terminal and advance the conversation.

[1583] 3. Closing proposal:

[1584] Based on the flow of the conversation, make a closing proposal. For example, you might suggest, "Our A-brand refrigerators use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[1585] 4. Emotional monitoring:

[1586] During the conversation, the user's facial expressions and voice tone are analyzed in real time, and that data is sent to the server.

[1587] 5. Feedback received:

[1588] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[1589] Specific example:

[1590] Examples of prompts to input into a generative AI model include the following:

[1591] Please select the "Consultation on purchasing home appliances" scenario. The initial message from the customer is "Hello, I'm thinking about purchasing home appliances." Next, respond with "What kind of products are you looking for?" If the user's expression seems anxious, make the customer's response more positive. Finally, make a closing suggestion by saying, "Here are some products I recommend."

[1592] In this way, by using this system, new sales staff can efficiently improve their customer service skills in a practical environment.

[1593] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1594] Step 1:

[1595] Scenario Selection

[1596] The server receives scenario information specified by the user via a scenario selection mechanism. The user selects the scenario they want to practice (for example, "consultation on purchasing home appliances") on the terminal's scenario selection screen. The input is scenario information, and the output is the system settings based on the selected scenario.

[1597] Step 2:

[1598] Acquisition of customer role information

[1599] The server uses a customer role information acquisition mechanism to retrieve relevant customer role information from the database based on the selected scenario. For example, past purchase history and attribute information of a customer considering purchasing home appliances. The input is a request for customer role information based on the scenario, and the output is the retrieved customer role information.

[1600] Step 3:

[1601] Scenario generation

[1602] The server generates a scenario from a template based on customer role information obtained via a scenario generation mechanism. For example, it might generate a scenario that includes the initial message, "Hello, I'm thinking of buying some home appliances." The input is customer role information, and the output is the generated scenario.

[1603] Step 4:

[1604] Receiving user input

[1605] The terminal displays customer-facing messages based on a scenario and accepts user input in real time. For example, the user might input, "What product are you looking for?" The input is the user's response, and the output is a record of that response.

[1606] Step 5:

[1607] Customer response generation

[1608] The server uses a customer response generation mechanism to analyze user input and generate the next customer response based on that input. For example, it might generate a response like, "I'm looking at refrigerators. Could you tell me the difference between products from company A and company B?" The input is the user's response, and the output is the generated customer response.

[1609] Step 6:

[1610] Saving conversation logs

[1611] The server uses a conversation log storage mechanism to save logs of virtual conversations between the user and the customer. The saved content includes the user's statements, the customer's generated responses, and timestamps. The input is each sequence of the conversation, and the output is the saved log.

[1612] Step 7:

[1613] Emotion analysis and transmission

[1614] The device uses a camera and microphone to capture the user's facial expressions and voice tone in real time, and analyzes them using emotion recognition technology. The analyzed emotion data is sent to a server. The input is the user's facial expressions and voice, and the output is the analyzed emotion data.

[1615] Step 8:

[1616] Audio output of the customer's response

[1617] The server outputs the customer's responses, generated using speech synthesis, as audio. This allows the user to train in more realistic scenarios. The input is the generated customer's responses, and the output is the audio response.

[1618] Step 9:

[1619] Generating and displaying feedback

[1620] The server generates an evaluation and feedback on the user's closing proposal using a feedback generation mechanism, based on the conversation log and analyzed sentiment data. This is then sent to the terminal using an evaluation display mechanism and displayed to the user. For example, feedback such as "This was a very clear closing proposal, but it lacked specific pricing" might be displayed. The input is the conversation log and sentiment data, and the output is the generated feedback.

[1621] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1622] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1623] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1624] [Fourth Embodiment]

[1625] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1626] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1627] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1628] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1629] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1630] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1631] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1632] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1633] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1634] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1635] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1636] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1637] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1638] The system of this invention is designed to provide a practice environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users (new employees), and its specific operation is described below.

[1639] server

[1640] The server plays a central role in generating scenarios and customer responses.

[1641] 1. Scenario selection method:

[1642] The server receives the scenario selection information specified by the user.

[1643] Specific example: The user selects the scenario "consultation on purchasing home appliances."

[1644] 2. Means of obtaining customer role information:

[1645] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[1646] Specific example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[1647] 3. Scenario generation method:

[1648] Based on the acquired customer information, a scenario is generated from a pre-prepared template.

[1649] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[1650] 4. User input receiving means:

[1651] The server receives user input from the terminal.

[1652] Specific example: Receive information entered by the user, such as "What kind of product are you looking for?".

[1653] 5. Means for generating customer role responses:

[1654] The server analyzes the received user input and generates the next response from the customer based on that input.

[1655] Specific example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[1656] 6. Means of saving conversation logs:

[1657] The server saves logs of virtual conversations between users and customers.

[1658] Specific example: Save each exchange in a conversation in chronological order.

[1659] 7. Feedback generation means:

[1660] Based on the conversation log, it generates an evaluation and feedback on the user's closing suggestion.

[1661] Specific example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1662] 8. Means of displaying evaluations:

[1663] The server sends the generated feedback to the terminal and displays it to the user.

[1664] Specific example: Display the feedback content on the user's device.

[1665] terminal

[1666] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[1667] 1. Scenario selection screen:

[1668] The user is prompted to select a scenario on the initial screen.

[1669] Specific example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[1670] 2. User input screen:

[1671] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[1672] Specific example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[1673] 3. Response display screen:

[1674] Displays the customer's response to the user's input.

[1675] Specific example: The server displays the response, "I am considering purchasing a refrigerator. Please tell me the differences between products from company A and company B."

[1676] 4. Feedback screen:

[1677] After the conversation ends, the feedback sent from the server will be displayed.

[1678] Specific example: Display a review stating, "The closing proposal was very clear, but it lacked a specific price."

[1679] User

[1680] Users operate this system to simulate interactions with actual customers.

[1681] 1. Scenario Selection:

[1682] The user selects the scenario they want to practice from the scenario selection screen on their device.

[1683] Example: Select "Consultation on purchasing home appliances".

[1684] 2. The progress of the virtual conversation:

[1685] The user progresses the conversation by responding to messages from a customer displayed on their device.

[1686] Example: Continue the conversation with the customer by typing, "What kind of product are you looking for?"

[1687] 3. Closing proposal:

[1688] Based on the flow of the conversation, the user makes a closing proposal.

[1689] Specific example: "Our refrigerators from Company A use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[1690] 4. Feedback received:

[1691] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[1692] Specific example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[1693] In this way, by using this system, new employees can efficiently improve their closing proposal skills in a practical environment.

[1694] The following describes the processing flow.

[1695] Step 1:

[1696] The user opens the scenario selection screen using their device and specifies the scenario they want to practice. The scenario selection information is sent to the server.

[1697] Step 2:

[1698] The server analyzes the scenario selection information received from the user and retrieves customer role information related to the selected scenario from the database.

[1699] Step 3:

[1700] Based on the customer information acquired by the server, a scenario is generated from a pre-prepared template. The generated scenario data is then sent to the terminal.

[1701] Step 4:

[1702] Based on the scenario data received from the server, the terminal displays an initial message to the user, who is playing the role of a customer. The user then enters a response to the initial message.

[1703] Step 5:

[1704] User input is sent from the terminal to the server. The server analyzes the user input and generates the next customer response based on its content.

[1705] Step 6:

[1706] The server sends the next response generated by the customer role to the terminal. The terminal displays this to the user. The user then enters their response again.

[1707] Step 7:

[1708] Steps 5 and 6 are repeated, and a virtual conversation between the user and the customer progresses. The user makes closing suggestions based on the flow of the conversation and continues to input.

[1709] Step 8:

[1710] Once the conversation ends, the terminal sends all conversation logs to the server. The server saves the conversation logs and begins analysis.

[1711] Step 9:

[1712] The server generates an evaluation and feedback on the user's closing suggestion based on the conversation log. The generated feedback is then sent to the terminal.

[1713] Step 10:

[1714] The device displays feedback received from the server to the user. The user reviews the feedback and uses it to improve their next practice session.

[1715] (Example 1)

[1716] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1717] Providing an effective training environment for new employees to practically improve their closing proposal skills is challenging. Traditional training lacks real-time feedback and practice using concrete scenarios, hindering the development of skills relevant to actual work. To address this challenge, a system is needed that provides real-time quantitative and qualitative feedback through virtual customer interactions.

[1718] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1719] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a user input means via a terminal, a scenario display means via a terminal, a customer role response display means via a terminal, a feedback display means via a terminal, a means for acquiring customer role information from a database via the server, a scenario generation means via the server, a user input analysis means via the server, a customer role message display means via a terminal, and a means for generating and displaying evaluations and feedback based on the conversation log. This enables the user to effectively practice dialogue skills and closing proposal skills through real-time virtual conversations based on scenarios and to receive detailed feedback.

[1720] A "scenario selection method" is a function that provides an interface for users to select the scenario they want to practice.

[1721] The "customer role information acquisition method" is a function that retrieves information about customer roles related to a scenario from a database.

[1722] The "scenario generation method" is a function that generates a scenario using a template based on the acquired customer role information.

[1723] A "user input receiving means" is a function that receives input from the user and analyzes it.

[1724] The "customer role response generation means" is a function that generates the next response of the customer role based on user input.

[1725] The "conversation log saving method" is a function that saves a chronological log of a virtual conversation between the user and the customer.

[1726] The "feedback generation method" is a function that generates evaluations and feedback on the user's closing suggestion based on the saved conversation log.

[1727] "Evaluation display means" refers to a function that displays the generated feedback to the user.

[1728] "User input means via terminal" refers to a function that accepts user input in real time.

[1729] "Scenario display means via terminal" refers to a function that displays the generated scenario to the user.

[1730] "Means for displaying customer responses via a terminal" refers to a function that displays the customer's response to user input.

[1731] "Means of displaying feedback via the terminal" refers to a function that displays the generated feedback to the user after the conversation has ended.

[1732] "Means of retrieving customer role information from the database by the server" refers to the function of retrieving customer role information from the database.

[1733] "Server-based scenario generation method" refers to a function that generates scenarios using templates based on customer information.

[1734] "Server-based user input analysis means" refers to a function that analyzes user input and generates the next customer response based on that analysis.

[1735] "Means for displaying customer-role messages via a terminal" refers to a function that displays messages from the customer role to the user.

[1736] "Means for generating and displaying evaluations and feedback based on conversation logs" refers to a function that generates evaluations and feedback on a user's closing suggestion based on saved conversation logs and displays them to the user.

[1737] The present invention provides a training environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users, and each component works in cooperation with the others.

[1738] System Configuration

[1739] server

[1740] The server plays a central role in the system, providing the means to generate scenarios and customer responses. The server has the following functions:

[1741] 1. Scenario Selection Method

[1742] The server receives scenario selection information specified by the user.

[1743] Example: The user selects the scenario "Consultation on purchasing home appliances".

[1744] 2. Means for obtaining customer role information

[1745] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[1746] Example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[1747] 3. Scenario generation means

[1748] The server generates a scenario from a template based on the customer role information it has acquired.

[1749] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[1750] 4. User input receiving means

[1751] The server receives user input from the terminal.

[1752] Example: Receive information entered by the user, such as "What kind of product are you looking for?".

[1753] 5. Customer response generation means

[1754] The server analyzes the user's input and generates the next customer's response based on that input.

[1755] Example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[1756] 6. Means of saving conversation logs

[1757] The server saves logs of virtual conversations between users and customers.

[1758] Example: Save all conversations in chronological order.

[1759] 7. Feedback generation means

[1760] The server generates an evaluation and feedback on the user's closing suggestion based on the saved conversation logs.

[1761] Example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1762] 8. Evaluation display means

[1763] The server sends the generated feedback to the terminal and displays it to the user.

[1764] Example: Display the feedback content on the user's device.

[1765] terminal

[1766] The terminal provides an interface for users to engage in virtual conversations through communication with the server. The terminal's functions are as follows:

[1767] 1. Scenario Selection Screen

[1768] The user is prompted to select a scenario on the initial screen.

[1769] Example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[1770] 2. User input screen

[1771] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[1772] Example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[1773] 3. Response display screen

[1774] Displays the customer's response to the user's input.

[1775] Example: The server will display the response, "I am considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[1776] 4. Feedback screen

[1777] After the conversation ends, the feedback sent from the server will be displayed.

[1778] Example: Display a review such as, "It was a very clear closing proposal, but it lacked a specific price."

[1779] User

[1780] The user operates this system to simulate interactions with actual customers. Follow these steps:

[1781] 1. Scenario Selection

[1782] The user selects the scenario they want to practice from the scenario selection screen on their device.

[1783] Example: Select "Consultation on purchasing home appliances".

[1784] 2. Progress of the virtual conversation

[1785] The user progresses the conversation by responding to messages from a customer displayed on their device.

[1786] Example: Continue the conversation with the customer by typing "What kind of product are you looking for?"

[1787] 3. Closed proposal

[1788] Based on the flow of the conversation, the user makes a closing proposal.

[1789] Example: "Our A Company refrigerators use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[1790] 4. Receiving Feedback

[1791] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[1792] Example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[1793] Example of a prompt

[1794] "Analyze the following user input and generate an appropriate response to the virtual customer's question: 'What products are you looking for?'"

[1795] In this way, by using this system, new employees can efficiently improve their closing proposal skills in a practical environment.

[1796] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1797] Step 1: Scenario Selection

[1798] The server receives scenario selection information sent from the terminal. Based on this input information, the server recognizes the selected scenario and prepares for the next processing. For example, if a user selects the scenario "consultation on purchasing home appliances," that information is sent to the server.

[1799] Step 2: Obtain customer role information

[1800] The server uses the scenario selection information to retrieve relevant customer role information from the database. This process involves executing database queries and extracting the necessary data. The retrieved data includes customer attribute information and past sales history. For example, the server retrieves attribute information and past sales history for a "customer considering purchasing home appliances."

[1801] Step 3: Scenario Generation

[1802] The server generates a scenario using a template based on the customer role information it has acquired. In this process, data is applied to the template to generate a specific conversation scenario. The generated scenario includes an initial message and questions. For example, the server generates a scenario based on "consultation about purchasing home appliances" and creates an initial message such as "Hello, I'm thinking about purchasing home appliances."

[1803] Step 4: Display the scenario

[1804] The terminal displays the scenario sent from the server to the user. In this process, the terminal converts the received data into an appropriate layout for display on the screen. It receives scenario data from the server as input and displays it to the user as output. For example, the terminal displays the message "Hello, I'm thinking about buying home appliances" to the user.

[1805] Step 5: User response input

[1806] The user enters a response to a customer-facing message displayed on the terminal. The entered data is sent from the terminal to the server. For example, the user might enter "What product are you looking for?"

[1807] Step 6: Receiving and analyzing user input

[1808] The server analyzes user input received from the terminal. This process uses natural language processing techniques to understand the user's intent and generate the next customer response. It receives the user's message as input, analyzes it, and then generates the next response data. For example, the server analyzes the input "What kind of product are you looking for?" and generates the response "I'm considering a refrigerator. Could you tell me the difference between products from company A and company B?"

[1809] Step 7: Customer's reaction

[1810] The terminal displays customer responses generated by the server to the user. In this process, it receives response data from the server and displays it appropriately on the screen. It receives response data as input and displays it to the user as output. For example, the terminal displays, "I'm considering buying a refrigerator. Could you tell me the differences between products from company A and company B?"

[1811] Step 8: Saving the conversation log

[1812] The server saves logs of virtual conversations between the user and the customer. This process stores all conversational exchanges chronologically in a database. It takes conversational data as input and saves it to the database as output. For example, the server saves each individual exchange in a conversation as a log.

[1813] Step 9: Generating Feedback

[1814] The server generates an evaluation and feedback on the user's closing proposal based on the saved conversation logs. This process analyzes the conversation logs and creates feedback based on evaluation criteria. It takes conversation logs as input and generates feedback data as output. For example, the server might generate feedback such as, "This was a very clear closing proposal, but it lacked a specific price offer."

[1815] Step 10: Displaying Feedback

[1816] The terminal displays feedback sent from the server to the user. This process converts the feedback data into an appropriate layout for display on the screen. It receives feedback data as input and displays it to the user as output. For example, the terminal displays the feedback, "It was a very clear closing offer, but it lacked specific pricing information."

[1817] (Application Example 1)

[1818] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1819] For new employees to efficiently improve skills such as presentations and content proposals, practice in a realistic environment is necessary. However, many companies have limited employees and resources for such practice, making it difficult to provide effective training. Furthermore, there are challenges such as limited opportunities for individual feedback and difficulty in identifying areas for improvement through self-assessment.

[1820] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1821] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a response training means including scenario selection information, and a prompt sentence generation means. This allows the user to practice in real time through interaction with a virtual customer and receive individualized feedback.

[1822] "Scenario selection method" refers to the means by which users select a scenario for practice.

[1823] "Method for obtaining customer role information" refers to a means for obtaining information about customer roles related to the selected scenario from a database.

[1824] A "scenario generation method" is a means of generating a scenario from a pre-prepared template based on acquired customer role information.

[1825] "User input receiving means" refers to means for receiving user input.

[1826] "Customer role response generation means" refers to means for generating the next response of the customer role based on the received user input.

[1827] A "conversation log saving method" is a means for saving logs of virtual conversations between a user and a customer.

[1828] A "feedback generation method" is a means of generating evaluations and feedback on a user's closing suggestion based on conversation logs.

[1829] "Evaluation display means" refers to a means for displaying the generated feedback to the user.

[1830] A "response training method including scenario selection information" is a means for training responses based on scenario selection information specified by the user.

[1831] A "prompt message generation means" is a means for generating prompt messages that users will use.

[1832] The present invention is a system for efficiently improving the presentation skills and content proposal abilities of new employees. This system includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, a response training means including scenario selection information, and a prompt sentence generation means.

[1833] server

[1834] Scenario selection method

[1835] The server receives information from the user about the scenario they want to practice. For example, if the user selects "New Product Presentation Scenario," the server processes this information.

[1836] Customer information acquisition method

[1837] The server retrieves customer role information from the database based on the selected scenario. This information includes customer profiles and past conversation history.

[1838] Scenario generation method

[1839] Based on the customer information obtained, the server generates an appropriate scenario from pre-prepared templates. For example, it might generate a "presentation scenario emphasizing the advantages of a new product."

[1840] User input receiving means

[1841] The server receives user input (for example, a question like, "How is the new product better?") in real time.

[1842] Customer response generation means

[1843] The server analyzes user input and generates the next response from the customer role based on that input. For example, in response to user input, it might generate a response such as, "The new product incorporates the latest technology and is very easy to use."

[1844] Conversation log saving method

[1845] The server stores a chronological log of the conversation between the user and the customer. This log is used later to generate feedback.

[1846] Feedback generation means

[1847] Based on the conversation log, the server generates user closing suggestions and an overall evaluation and feedback on the presentation. For example, it might provide feedback such as, "The presentation was generally easy to understand, but it lacked detailed explanations of the specifications."

[1848] Evaluation display means

[1849] The server sends the generated feedback to the user's device, allowing the user to check their performance.

[1850] terminal

[1851] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[1852] Scenario selection screen

[1853] The initial screen allows the user to select a scenario. For example, a screen might be displayed where the user can select "New Product Presentation Scenario."

[1854] User input screen

[1855] It displays an initial message from a customer based on a scenario and accepts user input in real time. For example, it might display the message, "Hello, please give me a presentation on the new product," and then receive a response from the user.

[1856] Response display screen

[1857] It displays the customer's response to the user's input. For example, it might display the server's response: "The new product incorporates the latest technology and is very easy to use."

[1858] Feedback screen

[1859] After the conversation ends, the server displays the feedback it has received. For example, it might show an evaluation such as, "The presentation was generally easy to understand, but it lacked detailed explanations of the specifications."

[1860] User

[1861] Scenario Selection

[1862] The user selects the scenario they want to practice from the scenario selection screen on their device. For example, they might select "New Product Presentation Scenario."

[1863] Progress of the virtual conversation

[1864] The user progresses the conversation by responding to messages from a customer displayed on the device. For example, they might type, "How is the new product better?" to continue the conversation.

[1865] Closed proposal

[1866] Based on the flow of the conversation, the user makes a closing suggestion. For example, they might suggest, "This new product is reasonably priced, so please give it a try."

[1867] Feedback received

[1868] After the conversation ends, review the feedback displayed on the device and use it to improve your next practice session. For example, you might receive feedback such as, "The presentation was generally easy to understand, but there was a lack of detailed explanation of the specifications," and use that to identify areas for improvement.

[1869] Specific examples and prompt statements

[1870] This example illustrates how new content creators can learn to present new content. An example of a prompt is: "Our new content offers clear visuals and intuitive controls that will excite viewers. Is there anything specific you're concerned about?" In this way, newcomers can effectively improve their skills by utilizing scenarios provided by the server and real-time feedback.

[1871] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1872] Step 1:

[1873] The server receives scenario selection information from the user through a scenario selection mechanism. Specifically, the user selects the scenario they want to practice and sends that information to the server. The input is scenario selection information, and the output is the selected scenario information.

[1874] Step 2:

[1875] The server uses a customer role information acquisition mechanism to retrieve customer role information related to the selected scenario from the database. This includes data such as the customer role's profile and past conversation history. The input is the selected scenario information, and the output is the retrieved customer role information.

[1876] Step 3:

[1877] The server uses a scenario generation mechanism to generate a scenario based on the acquired customer role information. A specific scenario is created using a template. The input is customer role information and a template, and the output is the generated scenario.

[1878] Step 4:

[1879] The server receives user input through a user input receiving mechanism. The user inputs questions and suggestions for the customer role based on a scenario. The input is direct information from the user, and the output is that same input information.

[1880] Step 5:

[1881] The server uses a customer role response generation mechanism to analyze the received user input and generates the next customer role response using a generation AI model. The input is the user's input information, and the output is the generated customer role response.

[1882] Step 6:

[1883] The server uses a conversation log storage mechanism to save logs of virtual conversations between the user and the customer. The input consists of the user's input information and the customer's responses, and the output is the saved conversation log.

[1884] Step 7:

[1885] The server uses a feedback generation mechanism to generate an evaluation and feedback on the user's closing suggestion based on the saved conversation log. The input is the saved conversation log, and the output is the generated feedback information.

[1886] Step 8:

[1887] The server displays the generated feedback to the user through the evaluation display means. The feedback content is displayed on the user's terminal. The input is the feedback information, and the output is the feedback displayed on the user's terminal.

[1888] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1889] The system of this invention is designed to provide a practice environment for effectively improving the closing proposal skills of new employees. This system consists of a server, terminals, and users (new employees), and its specific operation is described below.

[1890] server

[1891] The server plays a central role in generating scenarios and customer responses. It also uses an emotion engine to analyze user emotions and reflect them in the system's responses.

[1892] 1. Scenario selection method:

[1893] The server receives the scenario selection information specified by the user.

[1894] Specific example: The user selects the scenario "consultation on purchasing home appliances."

[1895] 2. Means of obtaining customer role information:

[1896] Based on the selected scenario, the server retrieves relevant customer role information from the database.

[1897] Specific example: Retrieve customer attributes and past sales history from a database of customers considering purchasing home appliances.

[1898] 3. Scenario generation method:

[1899] Based on the acquired customer information, a scenario is generated from a pre-prepared template.

[1900] Example: Based on a scenario for consulting about purchasing home appliances, generate initial messages and questions for the customer.

[1901] 4. User input receiving means:

[1902] The server receives user input from the terminal.

[1903] Specific example: Receive information entered by the user, such as "What kind of product are you looking for?".

[1904] 5. Means for generating customer role responses:

[1905] The server analyzes the received user input and generates the next response from the customer based on that input.

[1906] Specific example: In response to user input, the system generates the message, "I'm considering buying a refrigerator. Please tell me the differences between products from company A and company B."

[1907] 6. Means of saving conversation logs:

[1908] The server saves logs of virtual conversations between users and customers.

[1909] Specific example: Save each exchange in a conversation in chronological order.

[1910] 7. Feedback generation means:

[1911] Based on the conversation log, it generates an evaluation and feedback on the user's closing suggestion.

[1912] Specific example: Generate feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[1913] 8. Means of displaying evaluations:

[1914] The server sends the generated feedback to the terminal and displays it to the user.

[1915] Specific example: Display the feedback content on the user's device.

[1916] 9. Emotional Engine:

[1917] The server uses an emotion engine to analyze the user's facial expressions and tone of voice to recognize their emotions.

[1918] Specific example: If a user shows an expression of dissatisfaction, analyze that emotional data and generate a scenario that includes positive information.

[1919] terminal

[1920] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[1921] 1. Scenario selection screen:

[1922] The user is prompted to select a scenario on the initial screen.

[1923] Specific example: Display a screen to select the "consultation on purchasing home appliances" scenario.

[1924] 2. User input screen:

[1925] It displays initial messages from the customer role based on a scenario and accepts user input in real time.

[1926] Specific example: Display the message "Hello, I'm thinking of buying a home appliance" to the user and receive a response.

[1927] 3. Response display screen:

[1928] Displays the customer's response to the user's input.

[1929] Specific example: The server displays the response, "I am considering purchasing a refrigerator. Please tell me the differences between products from company A and company B."

[1930] 4. Emotion Recognition Interface:

[1931] The device uses cameras and microphones to analyze the user's facial expressions and voice tone.

[1932] Specific example: Sending the tone of voice and facial expressions of a user to the server in real time when they speak.

[1933] 5. Feedback screen:

[1934] After the conversation ends, the feedback sent from the server will be displayed.

[1935] Specific example: Display a review stating, "The closing proposal was very clear, but it lacked a specific price."

[1936] User

[1937] Users operate this system to simulate interactions with actual customers. Furthermore, an emotion engine analyzes those emotions in real time.

[1938] 1. Scenario Selection:

[1939] The user selects the scenario they want to practice from the scenario selection screen on their device.

[1940] Example: Select "Consultation on purchasing home appliances".

[1941] 2. The progress of the virtual conversation:

[1942] The user progresses the conversation by responding to messages from a customer displayed on their device.

[1943] Example: Continue the conversation with the customer by typing, "What kind of product are you looking for?"

[1944] 3. Closing proposal:

[1945] Based on the flow of the conversation, the user makes a closing proposal.

[1946] Specific example: "Our refrigerators from Company A use the latest energy-saving technology, which will save you money on electricity bills in the long run."

[1947] 4. Emotional monitoring:

[1948] During the conversation, the user's facial expressions and voice tone are analyzed in real time, and that data is sent to the server.

[1949] Specific example: If a user shows signs of anxiety, this is analyzed and influences the next scenario.

[1950] 5. Feedback received:

[1951] After the conversation ends, review the feedback displayed on your device and use it to improve your next practice session.

[1952] Specific example: We received feedback such as, "The closing proposal was very clear, but it lacked specific pricing information," and identified areas for improvement.

[1953] Thus, by using this system, new employees can not only efficiently improve their closing proposal skills in a practical environment, but also engage in more advanced practice that takes user emotions into consideration.

[1954] The following describes the processing flow.

[1955] Step 1:

[1956] The user opens the scenario selection screen using their device and specifies the scenario they want to practice. The scenario selection information is sent to the server.

[1957] Step 2:

[1958] The server analyzes the scenario selection information received from the user and retrieves customer role information related to the selected scenario from the database.

[1959] Step 3:

[1960] Based on the customer information acquired by the server, a scenario is generated from a pre-prepared template. The generated scenario data is then sent to the terminal.

[1961] Step 4:

[1962] Based on the scenario data received from the server, the terminal displays an initial message to the user, who is playing the role of a customer. The user then enters a response to the initial message.

[1963] Step 5:

[1964] User input is sent from the terminal to the server. The server analyzes the user input and generates the next customer response based on its content.

[1965] Step 6:

[1966] The server sends the next response generated by the customer role to the terminal. The terminal displays this to the user. The user then enters their response again.

[1967] Step 7:

[1968] The device uses its built-in camera and microphone to transmit the user's facial expressions and voice tone to the emotion engine in real time. The emotion engine analyzes this data to identify the user's emotional state.

[1969] Step 8:

[1970] The emotion engine recognizes the user's emotional state, which is then sent to the server. The server uses this information to adjust the customer's next response.

[1971] Step 9:

[1972] The server sends a customer's response, generated after considering emotional data, to the terminal. The terminal then displays this to the user.

[1973] Step 10:

[1974] The process from Step 5 to Step 9 is repeated, and a virtual conversation between the user and the customer progresses. The user makes closing suggestions based on the flow of the conversation and continues to input. In addition, the emotion engine monitors the user's emotional changes in real time.

[1975] Step 11:

[1976] Once the conversation ends, the terminal sends all conversation logs to the server. The server saves the conversation logs and begins analysis.

[1977] Step 12:

[1978] The server generates an evaluation and feedback on the user's closing suggestion based on the conversation log. The generated feedback is then sent to the terminal.

[1979] Step 13:

[1980] The device displays feedback received from the server to the user. The user reviews the feedback and uses it to improve their next practice session.

[1981] Through the above series of steps, users can improve their closing proposal skills through practical and emotionally responsive virtual conversations.

[1982] (Example 2)

[1983] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1984] Conventional systems for improving closing proposal skills for new employees suffer from monotonous customer responses, making it difficult to simulate the diverse reactions that occur in actual business negotiations. Furthermore, the lack of feedback that takes user emotions into account prevents effective practice that is relevant to real-world situations.

[1985] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, and an emotion analysis means. This enables the generation of diverse customer role responses that reflect the user's emotions, allowing practice in an environment close to an actual business negotiation.

[1986] "Scenario selection means" refers to a device or method that provides an interface for a user to select a practice scenario.

[1987] "Customer role information acquisition means" refers to a device or method for acquiring information about a virtual customer (customer role) from a database based on a selected scenario.

[1988] A "scenario generation means" is a device or method that generates a specific scenario by combining customer information with a pre-prepared template.

[1989] "User input receiving means" refers to a device or method for receiving content entered by a user from a terminal.

[1990] "Customer role response generation means" refers to a device or method that generates the next customer role response based on user input.

[1991] "Conversation log storage means" refers to a device or method for saving all logs of a virtual conversation.

[1992] "Feedback generation means" refers to a device or method that generates an evaluation and feedback on a user's closing suggestion based on a saved conversation log.

[1993] "Evaluation display means" refers to a device or method that displays the generated feedback on the user's terminal.

[1994] "Emotion analysis means" refers to a device or method for analyzing a user's facial expressions and tone of voice to recognize their emotions.

[1995] This invention is a system that provides a practice environment for effectively improving the closing proposal skills of new employees. The system consists of a server, terminals, and users, and their respective roles and operations are described below.

[1996] server

[1997] The server plays a central role in generating scenarios and customer responses to user input.

[1998] 1. Scenario Selection Method: The server receives scenario information specified by the user from the terminal. When the user selects "Consultation on purchasing home appliances," that information is sent to the server. The server then performs an appropriate database query based on this information.

[1999] 2. Means for obtaining customer role information: The server obtains customer role information from the database based on the selected scenario. For example, in the "consultation on purchasing home appliances" scenario, past sales history and customer attribute information are obtained using SQL queries.

[2000] 3. Scenario generation method: The server combines the acquired customer role information with templates to generate a specific scenario. For example, it creates a specific scenario such as "The customer is interested in refrigerators from companies A and B."

[2001] 4. User Input Reception Method: The server receives the information entered by the user from the terminal in real time. When the user enters "What product are you looking for?", that information is sent to the server and received by the server.

[2002] 5. Customer Role Response Generation Means: The server analyzes the received user input and generates the next customer role response based on its content. It uses a generation AI model to respond appropriately to the user's questions.

[2003] 6. Conversation log storage method: The server will save all logs of the virtual conversation. Each step of the conversation will be saved chronologically in the database for later reference.

[2004] 7. Feedback generation mechanism: The server generates an evaluation and feedback on the user's closing proposal based on the conversation log. It analyzes the log and provides feedback such as, "It was a very clear closing proposal, but it lacked specific pricing information."

[2005] 8. Evaluation display means: The server sends the generated feedback to the user's terminal for display.

[2006] 9. Emotion Analysis Method: The server analyzes the user's facial expressions and tone of voice to recognize their emotions. It also analyzes data obtained from the camera and microphone and incorporates it into the scenario as needed.

[2007] terminal

[2008] The terminal provides an interface for users to engage in virtual conversations through communication with the server.

[2009] 1. Scenario Selection Screen: The device will prompt the user to select a scenario on the initial screen. It will display multiple scenarios, such as "Consultation on purchasing home appliances," and wait until the user makes a selection.

[2010] 2. User Input Screen: The terminal displays an initial message for the customer role based on a scenario and accepts user input in real time. For example, it might display the message, "Hello, I'm thinking of buying a home appliance."

[2011] 3. Response Display Screen: The terminal displays the customer's response to the user's input. It displays the server's response: "I'm considering buying a refrigerator. Please tell me the differences between Company A's and Company B's products."

[2012] 4. Emotion Recognition Interface: The device uses a camera and microphone to analyze the user's facial expressions and voice tone. It captures the user's face and voice in real time and sends it to the server.

[2013] 5. Feedback screen: After the conversation ends, the device displays the feedback sent from the server. It displays the evaluation: "It was a very clear closing offer, but there was a lack of specific pricing information."

[2014] User

[2015] Users operate this system to simulate interactions with actual customers. Furthermore, an emotion engine analyzes those emotions in real time.

[2016] 1. Scenario Selection: The user selects the scenario they want to practice from the scenario selection screen on their device. Select "Consultation on purchasing home appliances" to start the scenario.

[2017] 2. Virtual Conversation Progression: The user progresses the conversation by responding to messages from the customer displayed on the terminal. For example, the user might type "What product are you looking for?" and continue the conversation with the customer.

[2018] 3. Closing Proposal: The user makes a closing proposal based on the flow of the conversation. For example, they might suggest, "Our A Company refrigerator uses the latest energy-saving technology and can reduce your electricity bill in the long run."

[2019] 4. Emotion Monitoring: The user's facial expressions and tone of voice are analyzed in real time and sent to the server. For example, if the user shows an anxious expression, this is analyzed and influences the next scenario.

[2020] 5. Receiving Feedback: After the conversation ends, the user reviews the feedback displayed on their device and uses it to improve their next practice session. For example, they might receive feedback such as, "The closing offer was very clear, but there was a lack of specific pricing information."

[2021] Usage examples and prompt messages

[2022] Usage example

[2023] The user selects the "consultation on purchasing home appliances" scenario on their device, and the conversation begins.

[2024] The first message displayed is, "Hello, I'm thinking of buying some home appliances."

[2025] The user types, "What kind of product are you looking for?", and the conversation proceeds.

[2026] The server generates a customer response saying, "I'm considering buying a refrigerator. Could you tell me the difference between products from company A and company B?" and sends it to the terminal.

[2027] A user suggests, "Our A-brand refrigerator uses the latest energy-saving technology, which will save you money on electricity bills."

[2028] Based on the conversation log and sentiment analysis results, the server generates feedback stating, "This was a very clear closing offer, but it lacked specific pricing information," and displays it on the terminal.

[2029] Example of a prompt

[2030] "Next scenario: Inquiry about purchasing home appliances. Please generate an initial message for the customer."

[2031] "Analyze the user's input and generate the next customer's response."

[2032] "Generate feedback and evaluation for the user's closing suggestion."

[2033] This system allows new employees to efficiently improve their closing proposal skills in a practical environment. Furthermore, the emotion engine enables advanced practice that takes user emotions into account.

[2034] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2035] Step 1:

[2036] The server receives scenario selection information sent from the terminal. When the user selects "consultation on purchasing home appliances" on the terminal, the server receives that information and performs the appropriate database query. The input is the user's scenario selection information, and the output is the database query related to the scenario.

[2037] Step 2:

[2038] The server retrieves customer role information from the database based on the scenario selection. It uses SQL queries to retrieve customer attribute information and past sales history related to "consultation on purchasing home appliances." The input is a database query, and the output is specific information about the customer role.

[2039] Step 3:

[2040] The server generates a scenario using a template based on the acquired customer role information. For example, it creates a specific scenario such as "The customer is interested in refrigerators from companies A and B." The input is customer role information and a template, and the output is the specific scenario.

[2041] Step 4:

[2042] The server sends the generated scenario to the user's terminal and displays the initial message of the scenario. For example, it might display the message, "Hello, I'm thinking about buying some home appliances." The input is the generated scenario, and the output is the initial message displayed on the terminal.

[2043] Step 5:

[2044] The terminal accepts user input in real time. When a user types "What product are you looking for?", that information is sent to the server. The input is the user's message, and the output is the input data sent to the server.

[2045] Step 6:

[2046] The server analyzes user input and generates the next customer response based on that input. It uses a generative AI model to generate appropriate responses to user questions. For example, it might generate the response, "I'm looking at refrigerators. Could you tell me the differences between products from company A and company B?" The input is the user's input data, and the output is the generated customer response.

[2047] Step 7:

[2048] The server sends the generated customer response to the user's terminal for display. The terminal displays the content and prompts the user for the next action. The input is the generated customer response, and the output is the message displayed on the terminal.

[2049] Step 8:

[2050] The server saves logs of all virtual conversations. Each step of the conversation is saved chronologically in the database for later reference. The input is the conversation log data, and the output is the saved conversation log.

[2051] Step 9:

[2052] The server generates an evaluation and feedback on the user's closing proposal based on the conversation log. It analyzes the log and provides feedback such as, "This was a very clear closing proposal, but it lacked specific pricing." The input is the conversation log data, and the output is the feedback content.

[2053] Step 10:

[2054] The server sends the generated feedback to the user's terminal, which then displays the feedback. The input is the generated feedback, and the output is the evaluation and suggestions displayed on the terminal.

[2055] Step 11:

[2056] The device uses a camera and microphone to analyze the user's facial expressions and voice tone. It captures the user's face and voice in real time and sends it to the server. The input is data obtained from the camera and microphone, and the output is the analyzed data sent to the server.

[2057] Step 12:

[2058] The server uses emotion analysis tools to analyze the user's facial expressions and tone of voice, and recognizes their emotions. For example, if it detects an anxious facial expression or tone of voice, it makes positive changes to the next scenario based on the analysis results. The input is the analysis data sent from the terminal, and the output is the analysis results and the modified scenario.

[2059] (Application Example 2)

[2060] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2061] Traditional customer service training systems have made it difficult for new sales staff to effectively learn how to respond to real-world customer service scenarios. Furthermore, it has been challenging to conduct more realistic training by analyzing user emotions in real time and providing feedback based on that analysis. Therefore, there is a need for a system that can quickly and reliably improve the customer service skills of new sales staff.

[2062] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2063] In this invention, the server includes a scenario selection means, a customer role information acquisition means, a scenario generation means, a user input receiving means, a customer role response generation means, a conversation log storage means, a feedback generation means, an evaluation display means, an emotion recognition means that analyzes emotions in real time using a camera and a microphone, a means that transmits information including the analyzed emotion data to the server, and a means that outputs the customer role's response as voice by speech synthesis. This makes it possible for new sales staff to practice customer service skills in a realistic manner and to effectively improve their skills through feedback that includes real-time emotion analysis.

[2064] A "scenario selection method" is a means for a user to select a specific scenario to use in training.

[2065] The "customer role information acquisition method" is a means of acquiring relevant customer role information from a database based on the selected scenario.

[2066] A "scenario generation method" is a method for generating a scenario from a template based on acquired customer role information.

[2067] "User input receiving means" refers to means for receiving user input.

[2068] "Customer role response generation means" refers to means for generating the next response of the customer role based on user input.

[2069] A "conversation log saving method" is a means of saving logs of virtual conversations between a user and a customer.

[2070] A "feedback generation method" is a means of generating evaluations and feedback on a user's closing suggestion based on conversation logs.

[2071] "Evaluation display means" refers to a means of sending the generated feedback to a terminal and displaying it to the user.

[2072] "Emotion recognition means" refers to a method of analyzing a user's emotions in real time using a camera and microphone.

[2073] "Means for transmitting information including analyzed emotion data to a server" refers to means for transmitting emotion data analyzed by emotion recognition means to a server.

[2074] "Means for outputting customer responses as audio using speech synthesis" refers to means for outputting the generated customer responses as audio.

[2075] The system for realizing this invention consists of three components: a server, a terminal, and a user. The functions and roles of each component are described below.

[2076] server

[2077] The server plays a central role in generating and managing scenarios, and receiving and analyzing user input. The server includes the following:

[2078] 1. Scenario selection method:

[2079] When a user selects a specific scenario to use for training, the scenario selection mechanism receives that information.

[2080] 2. Means of obtaining customer role information:

[2081] Based on the selected scenario, relevant customer information is retrieved from the database. For example, in a scenario involving a consultation about purchasing home appliances, past purchase history and customer attribute information are retrieved.

[2082] 3. Scenario generation method:

[2083] Based on the acquired customer information, a scenario is generated from a template. This scenario includes instructions on how the user should respond.

[2084] 4. User input receiving means:

[2085] It receives user input in real time. For example, it receives information such as "What kind of product are you looking for?" entered by the user.

[2086] 5. Means for generating customer role responses:

[2087] Based on user input, the system generates the next response from the customer. For ...

Claims

1. A scenario selection method for the user to select a virtual scenario for practice, A means for acquiring customer information, which retrieves information about a fictional character acting as a customer from a database, A scenario generation method that generates practice scenarios based on acquired customer role information, A user input receiving means for receiving input and responses from the user, A customer role response generation means that generates a virtual response of a customer role based on user input, A conversation log storage method for saving logs of virtual conversations between a user and a customer, A feedback generation means that analyzes saved conversation logs and generates feedback for the user, A means for displaying generated feedback and ratings to the user, A system that includes this.

2. The system according to claim 1, wherein the scenario generation means generates a scenario using a template.

3. The system according to claim 1, wherein the customer role response generation means generates the next response of the customer role based on user input.

Images