System

Abstract

A system is provided.SOLUTION: A system comprising: server means for providing a role-playing simulation for a user to learn customer service skills; means for a terminal to perform user login authentication; means for the server to verify user authentication; means for the terminal to display scenario options to the user; means for the server to prepare data based on the selected scenario; means for the terminal to display a scenario start screen; means for the server to proceed with the scenario and respond to inputs from the user; means for the server to generate feedback after the scenario ends; and means for the terminal to display the feedback to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional methods for acquiring customer service skills require repeated experience in actual customer service situations, making it difficult to obtain effective feedback and to fully develop the ability to respond to each situation. This results in the problem that it takes time to improve customer service skills and makes it difficult to maintain a consistent level of quality among store employees. Furthermore, since failure is not an option in actual customer service situations, training new employees is often difficult. [Means for solving the problem]

[0005] The present invention relates to a system that provides a role-playing simulation for users to learn customer service skills. This system includes the following means: terminal means for performing user login authentication, server means for verifying user authentication, terminal means for displaying scenario options, server means for preparing data based on the selected scenario, terminal means for displaying a screen for starting the scenario, server means for progressing through the scenario and responding to input from the user, server means for generating feedback after the scenario ends, and terminal means for displaying the feedback. This system allows users to efficiently improve their customer service skills in an environment that resembles an actual customer service situation, and to develop the ability to respond appropriately to each situation.

[0006] The "server means" is a computer server that provides role-play simulations for learning the user's customer service skills, manages scenarios, and generates feedback.

[0007] "Terminal means" refers to a device that allows a user to access the system, and provides an interface for entering login authentication, selecting a scenario, and displaying feedback.

[0008] "Login authentication" means the process of verifying a user's eligibility to access a system, including the entry and verification of a user ID and password.

[0009] "Scenario choices" refer to the multiple options provided to a user to select a particular customer service scenario to enact in a role-play.

[0010] "Scenario data" refers to information set in advance corresponding to each scenario, and includes product information, conversation templates, questions for the simulation, and the like.

[0011] "Scenario progression" is the process of responding to user input based on the selected scenario, and is the activity that manages the development of the scenario.

[0012] "User input analysis" is the process by which the server analyzes input from the user and generates an appropriate response or next action.

[0013] "Feedback generation" is the process by which the server provides evaluations and improvements based on the user's actions after the scenario is completed.

[0014] "Feedback display" is the process by which the terminal visualizes the feedback sent from the server to the user. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system that provides a role-play simulation for users to learn customer service skills. This system is composed of a server and a terminal, each of which operates as follows.

[0037] 1. Server startup and login authentication

[0038] The server starts the system and allows the user to access it. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters their authentication information, the server checks it against a database to confirm that they are a legitimate user. If authentication is successful, the user proceeds to the scenario selection screen.

[0039] 2. Scenario selection and preparation

[0040] The terminal displays a list of scenarios, and the user selects the one they are interested in. For example, if they select the "Handling Product Returns" scenario, the server loads the corresponding data (product information, conversation templates, simulation questions, etc.) from the database, and the scenario is ready to begin.

[0041] 3. Begin the role-play

[0042] The terminal displays a screen to start the scenario, and the server runs the scenario. The server generates a prompt such as "A customer wishes to return an item. How would you like to proceed?" and sends it to the terminal. When the user inputs "We would like to know the reason for the return," the server analyzes the input, generates a response from the customer, and sends it to the terminal. For example, a response such as "We will explain the reason. The item was damaged" is returned.

[0043] 4. Providing Feedback

[0044] When the scenario ends, the server analyzes the user's overall response and generates feedback. The feedback includes what went well and what needs improvement. For example, the server might say, "We properly confirmed the reason for the return, but please remember to suggest a coupon next time." The terminal displays this feedback for the user to review.

[0045] This system allows users to efficiently improve their customer service skills by simulating actual customer service situations. In particular, the feedback function allows users to identify specific areas for improvement and reflect them in their actual work.

[0046] The processing flow will be explained below.

[0047] Step 1:

[0048] The server starts the system. The server initializes the basic components of the role-play simulation system and prepares to generate the login screen.

[0049] Step 2:

[0050] The terminal displays a login screen to the user. The terminal provides the user with a form to enter their login ID and password, and includes a "Login" button.

[0051] Step 3:

[0052] The user enters their login credentials. The user enters their login ID and password and clicks the "Login" button.

[0053] Step 4:

[0054] The server receives the user's authentication information and verifies the login ID and password against the database.

[0055] Step 5:

[0056] The server sends the authentication result to the terminal. If the authentication is successful, the server sends an instruction to the terminal to display the scenario selection screen.

[0057] Step 6:

[0058] The terminal displays a scenario selection screen. The terminal presents the user with a list of scenarios, offering options such as "handling product returns."

[0059] Step 7:

[0060] The user selects a scenario: the user clicks on the scenario that interests them.

[0061] Step 8:

[0062] The server loads data based on the selected scenario, and retrieves relevant product information, conversation templates, simulation questions, etc. from the database.

[0063] Step 9:

[0064] The server notifies the terminal that it is ready to start the scenario. The server sends a command to the terminal to start the scenario.

[0065] Step 10:

[0066] The terminal displays the scenario start screen. The terminal displays a screen that explains the scenario's outline and purpose to the user.

[0067] Step 11:

[0068] The server runs the scenario. It generates a prompt such as "The customer wants to return the product. How would you like to proceed?" and sends it to the terminal.

[0069] Step 12:

[0070] The user enters a response, such as "We would like to ask you the reason for the return," in the chat box and presses the send button.

[0071] Step 13:

[0072] The server processes the user's input. The server analyzes the user's input and generates a response depending on the progress of the scenario.

[0073] Step 14:

[0074] The server sends a response to the terminal, for example, "I'll explain why. The product was damaged."

[0075] Step 15:

[0076] The terminal displays the response to the user. The terminal displays the response received from the server in a chat box.

[0077] Step 16:

[0078] When the scenario ends, the server generates feedback by analyzing the user's response history and creating feedback that includes what went well and what needs to be improved.

[0079] Step 17:

[0080] The server sends the feedback to the device. The server sends the generated feedback to the device.

[0081] Step 18:

[0082] The terminal displays the feedback. The terminal visualizes the feedback sent from the server to the user.

[0083] Example 1

[0084] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0085] Conventional customer service skill learning systems have difficulty simulating actual customer service situations, making it difficult to effectively improve the skills required for real-world work. In particular, it is difficult to generate appropriate responses from a customer role in real time in response to user input and provide feedback based on that, which limits the effectiveness of learning.

[0086] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0087] In this invention, the server includes means for providing a role-playing simulation for a user to learn customer service skills, means for the server to analyze user input using a natural language processing library, means for the server to generate responses for the customer role using a generative AI model, and means for the server to generate feedback using an analytical model after the scenario ends. This allows users to effectively improve their customer service skills through a simulation that resembles a real-life customer service situation, and further allows them to receive specific feedback on their responses.

[0088] A "user" is a person who uses the system to learn customer service techniques.

[0089] "Customer service skills" refers to the skills and knowledge required to deal with customers.

[0090] "Role-play simulation" is a system that allows users to practice by simulating actual customer service situations.

[0091] A "server" is a computer device that performs the main processing of a system.

[0092] A "terminal" is a computer device or device that allows a user to operate the system.

[0093] "Login authentication" is the procedure for verifying a user's authentication information and confirming their access rights to the system.

[0094] "Authentication information" refers to information used when logging in, such as a user ID and password.

[0095] A "scenario" is a pre-set story that simulates a specific customer service situation.

[0096] "Scenario Choices" are options that allow the user to select the scenario they want to practice.

[0097] A "natural language processing library" is a program library for analyzing human language.

[0098] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input.

[0099] A "prompt" is text that prompts the user for the next action or response.

[0100] A "database" is a system for organizing, storing, and retrieving data.

[0101] "Feedback" refers to an evaluation of the user's response and areas for improvement.

[0102] An "analytical model" is an algorithm or program used to evaluate a user's performance.

[0103] The present invention relates to a system that provides role-play simulations for users to learn customer service skills. This system is comprised of a server and a terminal, and operates as follows.

[0104] Server startup and login authentication

[0105] The server starts the system and establishes a connection with the database. For example, a backend server built on a cloud service (e.g., a Node.js server on AWS (registered trademark)) starts and connects to a MySQL (registered trademark) database. The terminal displays a login screen to the user, and the user enters a login ID and password. For example, the user enters "user123" and "pass123." The server receives the entered authentication information and executes an SQL query to compare it with the database. If authentication is successful, the server returns the data of the scenario selection screen to the terminal, which displays it.

[0106] Scenario selection and preparation

[0107] The terminal displays a list of scenarios to the user. When the user selects the "Handling Product Returns" scenario, the selection information is sent to the server. The server loads data corresponding to the selected scenario (e.g., detailed product information, conversation templates, simulation questions, etc.) from the database. For example, the query "SELECT FROM scenarios WHERE scenario_name="Handling Product Returns";" is used for this purpose. The server caches the loaded data, and when it is ready, displays the scenario start screen on the terminal.

[0108] Beginning the role-play

[0109] The device displays a scenario start screen to the user and presents the initial prompt. In response, for example, the text "A customer wishes to return an item. How would you like to proceed?" is displayed. The user enters "I would like to know the reason for the return" and presses the send button. The server receives the input and analyzes it using a natural language processing library (e.g., spaCy or NLTK). A generative AI model (e.g., GPT-3 (registered trademark)) is used to generate a response for the customer, such as "I will explain the reason. The item was damaged." The generated response is then sent to the device, which displays it.

[0110] Providing Feedback

[0111] After the scenario is over, the server analyzes the user's response. For example, it analyzes the user's response log and uses a generative AI model to perform an overall evaluation. Specific feedback is generated, such as a message like, "We've properly confirmed the reason for the return, but please remember to suggest a coupon next time." The terminal displays the generated feedback to the user, who can then review it.

[0112] In this way, the system of the present invention allows users to simulate customer service situations in an environment that closely resembles a real customer service situation, effectively improving their customer service skills. In addition, by receiving specific feedback, users can improve their own customer service methods and reflect them in their actual work.

[0113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0114] Step 1:

[0115] The server starts the system and establishes a connection to the database. The server starts a backend Node.js server and connects to the MySQL database. The system is now up and running and ready to accept user access.

[0116] Step 2:

[0117] The terminal displays a login screen and asks the user to enter authentication information. The user enters their login ID and password. Input: User's login ID and password. Output: User's authentication information is sent to the server.

[0118] Step 3:

[0119] The server checks the received authentication information against the database. The server executes an SQL query (e.g. SELECT FROM users WHERE user_id='user123' AND password='pass123';) to verify the authentication information. Input: User authentication information. Output: Authentication result (success or failure) is sent to the terminal.

[0120] Step 4:

[0121] The terminal displays the following screen to the user based on the authentication result. If authentication is successful, the terminal displays the scenario selection screen. Input: Authentication result. Output: Scenario selection screen.

[0122] Step 5:

[0123] The terminal displays a list of scenarios and lets the user select one. For example, the user selects the "Handling product returns" scenario. Input: User's scenario selection. Output: Selected scenario information is sent to the server.

[0124] Step 6:

[0125] The server loads data corresponding to the selected scenario from the database. For example, the server executes a query such as SELECT FROM scenarios WHERE scenario_name="Product return handling"; Input: Selected scenario information. Output: Corresponding scenario data is sent to the terminal.

[0126] Step 7:

[0127] The terminal displays the scenario start screen and presents the first prompt to the user. For example, the prompt may read, "A customer wishes to return an item. How would you like to proceed?" Input: Scenario data. Output: The first prompt is displayed to the user.

[0128] Step 8:

[0129] The user enters a response in the input box and submits it. For example, "Please tell us the reason for the return." Input: User's response. Output: User's response is sent to the server.

[0130] Step 9:

[0131] The server parses the user response using a natural language processing library, such as spaCy or NLTK. Input: User response. Output: Parsed data.

[0132] Step 10:

[0133] The server uses a generative AI model to generate a response for the customer. For example, it uses GPT-3 to generate a response like "I'll explain why. The item was damaged." Input: Parsed data. Output: Generated response for the customer.

[0134] Step 11:

[0135] The terminal displays the generated customer response to the user. Input: The generated customer response. Output: The customer response is displayed to the user.

[0136] Step 12:

[0137] After the scenario is completed, the server analyzes the user's responses and generates feedback. The analytical model is used to score the user's response log and identify areas for improvement. Input: User's response log. Output: Generated feedback.

[0138] Step 13:

[0139] The terminal displays the generated feedback to the user. For example, "We've properly verified the reason for the return, but don't forget to offer a coupon next time." Input: The generated feedback. Output: The feedback is displayed to the user.

[0140] This process allows users to simulate actual customer service situations and effectively learn customer service techniques. Through more specific responses and feedback, users can acquire skills that can be applied to actual work.

[0141] (Application example 1)

[0142] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Conventional role-play simulation systems have limited methods for providing appropriate feedback to users learning customer service skills. As a result, users experience insufficient learning through the simulation, making it difficult to identify specific areas for improvement that can be applied in practice. Furthermore, while conventional systems have a general feedback function, they lack dynamic and personalized feedback based on input responses. This makes it difficult for users to learn the optimal response methods for each specific scenario. Furthermore, basic functions such as login authentication and scenario selection have also presented issues in terms of security and convenience.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0145] In this invention, the server includes an information processing device that provides a role-playing simulation for a user to learn customer service skills; an operation terminal that performs user login authentication; an information processing device that verifies user authentication; an operation terminal that displays scenario options to the user; an information processing device that organizes data based on the selected scenario; an operation terminal that displays a screen indicating the start of the scenario; an information processing device that progresses through the scenario and responds to input from the user; an information processing device that generates feedback after the scenario ends; an operation terminal that displays the feedback to the user; a means for using a generative AI model to generate feedback based on user responses input from the operation terminal; and a means for inputting prompt sentences into the generative AI model. This allows the user to receive dynamic and personalized feedback in an interactive format and identify specific areas for improvement. This also provides high security and convenience, enabling effective and efficient improvement of customer service skills.

[0146] "User" refers to a person who uses the system to learn customer service skills, or the user of the system.

[0147] "Customer service skills" refer to the techniques and abilities required for dealing with customers, and are particularly required in customer service work in physical stores.

[0148] "Role-play simulation" refers to a simulation that simulates actual customer service work based on a specific scenario.

[0149] "Information processing device" refers to a computer device such as a server for processing and analyzing data, progressing a scenario, and the like.

[0150] An "operation terminal" is a device that is directly operated by a user, and includes mobile devices such as smartphones and tablets.

[0151] "Login authentication" refers to the process by which a user enters authentication information such as an ID and password to verify their identity in order to gain permission to access a system.

[0152] "User authentication" is the procedure in which the server verifies the authentication information entered by the user and confirms that the user is a legitimate user.

[0153] "Scenario options" refers to a list of multiple customer service scenarios that the user can experience, each of which simulates a specific situation.

[0154] "Preparing data" means preparing the information and settings required for the selected scenario and bringing it into a state ready for starting the simulation.

[0155] The "scenario start screen" refers to a screen that notifies the user of the start of a simulation.

[0156] A "generative AI model" is an artificial intelligence model that generates feedback and responses based on user input, and uses techniques such as machine learning.

[0157] A "prompt" is a question or instruction that tells the user what to do next.

[0158] The present invention relates to a system that provides role-play simulations for users to learn customer service skills. This system is composed of an information processing device (server) and an operation terminal (smartphone, tablet, etc.), and each operates as follows.

[0159] 1. User authentication:

[0160] The user accesses the system from an operation terminal and performs login authentication. When the user enters their ID and password, the information processing device verifies this and confirms that they are a legitimate user. If authentication is successful, the scenario selection screen is displayed.

[0161] 2. Scenario Selection:

[0162] The operation terminal displays multiple customer service scenarios (e.g., "handling product returns," "handling complaints," "product suggestions," etc.) to the user. When the user selects a scenario that interests them, the information processing device prepares the corresponding data (product information, conversation templates, simulation questions, etc.).

[0163] 3. Begin the role-play simulation:

[0164] The operating terminal displays a screen for starting the scenario, and interactive questions based on the scenario are displayed. The information processing device analyzes the user's input according to the progress of the scenario and generates an appropriate reply. For example, a prompt such as "A customer has complained about a defective product. How would you respond first?" is displayed. When the user enters a response, the information processing device analyzes it and generates the next response.

[0165] 4. Providing Feedback:

[0166] At the end of the scenario, the information processing device analyzes the user's overall response and generates feedback using a generative AI model. The feedback includes what was appropriate and what needs improvement. For example, specific advice such as "We properly confirmed the reason for the return, but don't forget to suggest a coupon next time" is provided. The operating device displays this feedback, allowing the user to confirm it.

[0167] Hardware and software used:

[0168] Hardware: smartphones, tablets, servers

[0169] Software: Python, API server, database, generative AI model

[0170] Examples:

[0171] The user opens the application and logs in by entering their ID and password. After logging in, they select the "Complaint Handling" scenario. The scenario begins and a prompt appears: "A customer has complained about a defective product. What is your first response?" The user enters their response and practices handling a complaint through a simulation. After the scenario ends, they review the provided feedback and improve their future responses.

[0172] In this way, users can efficiently improve their customer service skills by simulating actual customer service situations. Furthermore, interactive feedback allows users to identify specific areas for improvement and reflect them in their actual work.

[0173] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0174] Step 1:

[0175] (Login authentication)

[0176] The user enters an ID and password from the operating terminal. This is the input data. The terminal sends this authentication information to the server. The server verifies the authentication information against a database and confirms that the user is legitimate. If authentication is successful, the server generates a session token and sends it to the terminal. The terminal displays a scenario selection screen to the user. The output is the session token and the scenario selection screen.

[0177] Step 2:

[0178] (Scenario selection)

[0179] The user selects a scenario from the operating terminal. This is the input data. The terminal sends the selected scenario ID to the server. The server then prepares and retrieves the corresponding data (product information, conversation templates, simulation questions, etc.) from the database. The output is the prepared data.

[0180] Step 3:

[0181] (Scenario start screen display)

[0182] The terminal displays a scenario start screen to the user. A prompt sentence based on the scenario is displayed first. The input is the prepared data, and the output is the scenario start screen and the prompt sentence.

[0183] Step 4:

[0184] (Role-play simulation progress)

[0185] The user responds to the prompt sentence from the operating terminal. This response is the input data. The terminal sends this response to the server. The server uses a generative AI model to analyze the user's response and generate an appropriate reply. The generated reply is sent to the terminal and displayed to the user again. The output is the generated reply.

[0186] Step 5:

[0187] (Scenario end decision)

[0188] The server monitors the progress of the scenario and determines whether the scenario has finished. The progress of the scenario is the input data, and the decision on whether the scenario has finished is the output.

[0189] Step 6:

[0190] (Generating feedback)

[0191] Once the scenario is over, the server analyzes the user's overall response and generates feedback using a generative AI model, including what went well and what needs improvement. The input is the user's response log, and the output is the generated feedback.

[0192] Step 7:

[0193] (Show feedback)

[0194] The terminal displays the generated feedback to the user. The input is the generated feedback, and the output is the feedback display screen. The user can check the specific advice and improve their future actions.

[0195] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0196] The present invention relates to a system that provides a role-play simulation for users to learn customer service skills, and also combines it with an emotion engine that recognizes the user's emotions. This system is composed of a server means, a terminal means, and an emotion engine, each of which operates as follows.

[0197] 1. Server startup and login authentication

[0198] The server starts the system and initializes the basic components necessary to provide a learning environment for customer service skills. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters this authentication information, the server checks it against a database to confirm whether the user is a legitimate user. If authentication is successful, the scenario selection screen is displayed on the terminal.

[0199] 2. Scenario selection and preparation

[0200] The terminal displays a list of scenarios, and the user selects one that interests them, such as "Handling Product Returns." The server loads data corresponding to the selected scenario, such as product information, conversation templates, and simulation questions, from the database. Once the scenario is ready to start, that information is sent to the terminal.

[0201] 3. Role-play initiation and emotion recognition

[0202] The terminal displays a screen to start the scenario, and the server progresses the scenario. For example, a prompt such as "A customer wishes to return an item. How would you respond?" is generated and sent to the terminal. When the user inputs "I would like to know the reason for the return," the terminal sends the input to the emotion engine for analysis, which then sends the result to the server. Based on the analysis results of the emotion engine, the server generates a response for the customer and sends it to the terminal. For example, if the user inputs with an anxious look on their face, a response such as "I will explain the reason, but the item was damaged" is returned.

[0203] 4. Providing Feedback

[0204] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user handled the emotion appropriately and what areas need improvement. For example, specific advice such as "We properly confirmed the reason for the return, but next time, don't forget to offer a coupon. We recommend using more reassuring language to address the customer's concerns" may be provided. The device displays this feedback to the user, and by the user confirming it, the device can improve both customer service skills and emotional understanding skills.

[0205] This system allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. In particular, the addition of emotion recognition functionality allows users to learn how to respond to customers in accordance with their emotions, enabling them to provide high-quality service in real-world situations.

[0206] The processing flow will be explained below.

[0207] Step 1:

[0208] The server starts the system. The server initializes the basic components of the role-play simulation system and prepares to generate the login screen.

[0209] Step 2:

[0210] The terminal displays a login screen to the user. The terminal provides the user with a form to enter their login ID and password, and includes a "Login" button.

[0211] Step 3:

[0212] The user enters their login credentials. The user enters their login ID and password and clicks the "Login" button.

[0213] Step 4:

[0214] The server receives the user's authentication information and verifies the login ID and password against the database.

[0215] Step 5:

[0216] The server sends the authentication result to the terminal. If the authentication is successful, the server sends an instruction to the terminal to display the scenario selection screen.

[0217] Step 6:

[0218] The terminal displays a scenario selection screen. The terminal presents the user with a list of scenarios, offering options such as "handling product returns."

[0219] Step 7:

[0220] The user selects a scenario: the user clicks on the scenario that interests them.

[0221] Step 8:

[0222] The server loads data based on the selected scenario, and retrieves relevant product information, conversation templates, simulation questions, etc. from the database.

[0223] Step 9:

[0224] The server notifies the terminal that it is ready to start the scenario. The server sends a command to the terminal to start the scenario.

[0225] Step 10:

[0226] The terminal displays the scenario start screen. The terminal displays a screen that explains the scenario's outline and purpose to the user.

[0227] Step 11:

[0228] The server runs the scenario. It generates a prompt such as "The customer wants to return the product. How would you like to proceed?" and sends it to the terminal.

[0229] Step 12:

[0230] The user enters a response, such as "We would like to ask you the reason for the return," in the chat box and presses the send button.

[0231] Step 13:

[0232] The device sends the user's input to the emotion engine, which analyzes the input and tone of the words to identify the user's emotion.

[0233] Step 14:

[0234] The emotion engine sends the analysis results to the server, which include emotional information such as whether the user is anxious, angry, or relieved.

[0235] Step 15:

[0236] The server generates a response based on the user's emotional state: for example, if the user appears anxious, it generates a reassuring response such as, "The customer will explain why, but the item was damaged."

[0237] Step 16:

[0238] The server sends a response to the terminal. The generated response is sent to the terminal.

[0239] Step 17:

[0240] The terminal displays the response to the user. The terminal displays the response received from the server in a chat box.

[0241] Step 18:

[0242] When the scenario ends, the server generates feedback, summarizing the evaluation and areas for improvement based on the user's response history and the analysis results of the emotion engine.

[0243] Step 19:

[0244] The server sends the feedback to the terminal. The generated feedback is sent to the terminal.

[0245] Step 20:

[0246] The terminal displays the feedback. The terminal visualizes the feedback sent from the server to the user.

[0247] Example 2

[0248] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0249] In conventional customer service skill learning systems, it was difficult for users to learn appropriate responses based on actual customer emotional states. This resulted in problems such as insufficient acquisition of practical skills to improve on-site customer service skills and customer satisfaction. Furthermore, there was a lack of mechanisms to provide specific feedback on user behavior.

[0250] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0251] In this invention, the server includes computer means for providing a role-play simulation for a user to learn customer service skills, means for the terminal to perform login authentication of the user, means for the computer to verify user authentication, means for the terminal to display scenario options to the user, means for the computer to prepare data based on the selected scenario, means for the terminal to display a screen for starting the scenario, means for the computer to progress through the scenario and respond to input from the user, means for the terminal to send user input to an emotion engine, means for the emotion engine to analyze the user input and generate emotion data, means for the computer to generate a response based on the emotion analysis result, means for the computer to generate feedback after the scenario ends, and means for the terminal to display the feedback to the user. This enables the user to learn customer service skills tailored to the emotional states of actual customers and receive specific feedback.

[0252] "User" refers to a learner who uses the system to learn customer service skills.

[0253] "Computer means" refers to servers and cloud services that provide role-play simulations of customer service skills.

[0254] "Terminal" refers to a device such as a PC, tablet, or smartphone that a user uses to access the system.

[0255] "Login authentication" is the procedure for authenticating access to a system using a user's ID and password.

[0256] "User authentication verification" refers to the process of verifying whether the entered authentication information is legitimate.

[0257] "Scenario options" refers to multiple role-play scenarios that a user can choose from to learn.

[0258] "Data preparation" refers to a series of processes that load various information from a database based on a selected scenario.

[0259] The "scenario start screen" refers to the screen that is displayed to the user when the scenario starts.

[0260] "Scenario progression" refers to the process of managing the execution of the role-play simulation and the system's response to user input.

[0261] "Emotion engine" refers to software or algorithms that analyze user input and assess emotional state.

[0262] "Emotion data" refers to emotional information obtained by the emotion engine through analysis of user input.

[0263] "Response generation" refers to the process of creating an appropriate response to the user based on the analysis results of the emotion engine.

[0264] "Feedback" refers to information that provides an evaluation of the user's actions and decisions and specific advice after the scenario is completed.

[0265] "Displaying feedback" refers to presenting the generated feedback content to the user.

[0266] This invention relates to a system that provides a role-play simulation for users to learn customer service skills, and also combines it with an emotion engine that recognizes the user's emotions. The purpose of this system is to enable users to efficiently improve their customer service skills and emotional understanding in an environment that is similar to a real customer service situation. Specifically, the system is composed of a server, a terminal, and an emotion engine.

[0267] Server startup and login authentication

[0268] The server starts the system and initializes the basic components necessary to provide a learning environment for customer service skills. At this time, the server uses multiple components, including database connections, a user management system, and a scenario management system. Specifically, it uses a MySQL database, a Node.js server, and Nginx. The terminal provides the user with a login screen and prompts them to enter their login ID and password. When the user enters this authentication information, the server checks it against the database to confirm whether the user is a legitimate user. For example, it compares the hash values ​​of the user ID and password stored in the database. If authentication is successful, the scenario selection screen is displayed on the terminal.

[0269] Scenario selection and preparation

[0270] The device displays a list of scenarios, and the user selects the one they are interested in. For example, if the user selects a scenario such as "Handling Product Returns," the device sends the user's selection to the server. The server loads data corresponding to the selected scenario, such as product information, conversation templates, and simulation questions, from a database. At this time, the server efficiently retrieves the data using SQL queries, etc. Once the scenario is ready to start, that information is sent to the device. A framework such as React.js can be used by the device to display the scenario start screen.

[0271] Role-play initiation and emotion recognition

[0272] The device displays a screen to start the scenario, and the server runs the scenario. For example, a prompt such as "A customer wishes to return a product. How would you respond?" is generated and sent to the device. When generating this prompt, the server can use a generative AI model (e.g., OpenAI (registered trademark) GPT-3). When the user inputs "I would like to know the reason for the return," the device sends the input to an emotion engine for analysis. The emotion engine analyzes the input using a natural language processing API from Google (registered trademark) Cloud or similar and generates emotion data. For example, this emotion data may determine that the input expresses "anxiety." The analysis results are sent to the server, which generates a response based on the emotion analysis results and sends it to the device. For example, a response such as "I will explain the reason, but the product was damaged" may be returned.

[0273] Providing Feedback

[0274] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user appropriately responded to the user's emotions and what areas need improvement. For example, the server might provide specific advice such as, "We properly confirmed the reason for the return, but next time, please remember to offer a coupon. We recommend using more reassuring language to address the customer's concerns." The device displays this feedback to the user, and by allowing the user to review it, the server can improve both customer service skills and emotional understanding skills. When generating feedback, the server can use analytical tools such as Python's Pandas library. The device can also design the feedback screen to be visually appealing using HTML5 and CSS3.

[0275] This system allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. Specifically, by selecting a "product return handling" scenario and role-playing, users can practically learn how to respond to customer emotions. Furthermore, feedback provided after the system is completed provides users with specific guidelines for improving their own responses.

[0276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0277] Step 1: Server startup and login authentication

[0278] input:

[0279] The server receives the boot command and starts booting the system.

[0280] process:

[0281] The server initializes the basic components (e.g., database connection, user management system, scenario management system) required to provide a learning environment for customer service skills.

[0282] Specific initialization operations include connecting to the MySQL database, starting the Node.js server, and configuring Nginx.

[0283] output:

[0284] The server logs the completion of initialization, generates a login screen, and sends it to the terminal.

[0285] Step 2: Provide login authentication via device

[0286] input:

[0287] The terminal receives the login screen information from the server.

[0288] process:

[0289] The terminal displays a login screen and prompts the user to enter their login ID and password.

[0290] output:

[0291] The user enters their login ID and password, and the device sends the authentication information to the server.

[0292] Step 3: Verifying User Authentication with the Server

[0293] input:

[0294] The server receives the authentication information sent from the terminal.

[0295] process:

[0296] The server checks the user information in the database to verify that the entered authentication information is valid. Specifically, it compares the user ID and hashed password with the information in the database.

[0297] output:

[0298] If authentication is successful, the server generates a scenario selection screen and sends it to the terminal. If authentication fails, it sends an error message to the terminal.

[0299] Step 4: Providing scenario selection on the device

[0300] input:

[0301] The terminal receives the scenario selection screen information from the server.

[0302] process:

[0303] The terminal displays a list of scenarios and prompts the user to select a scenario.

[0304] output:

[0305] The user selects a scenario of interest (e.g., "handling product returns"), and the terminal transmits the selection information to the server.

[0306] Step 5: Prepare scenario data on the server

[0307] input:

[0308] The server receives the scenario selection information transmitted from the terminal.

[0309] process:

[0310] The server loads data corresponding to the selected scenario (e.g., product information, conversation templates, simulation questions) from the database. Specifically, it retrieves the relevant data using an SQL query based on the scenario ID.

[0311] output:

[0312] The server transmits the prepared scenario data to the terminal.

[0313] Step 6: Providing scenario start by terminal

[0314] input:

[0315] The terminal receives the scenario data from the server.

[0316] process:

[0317] The device displays a scenario start screen. Specifically, the scenario start screen is visually generated using a framework such as React.js.

[0318] output:

[0319] Notify the user that the scenario is ready to begin.

[0320] Step 7: Start the role-play and generate prompts with the server

[0321] input:

[0322] The user confirms the start of the scenario.

[0323] process:

[0324] The server runs the scenario and generates key prompts (e.g., "A customer wants to return an item. How would you handle this?") using a generative AI model (e.g., OpenAI GPT-3).

[0325] output:

[0326] The generated prompt is sent to the terminal.

[0327] Step 8: Prompting the Terminal and Getting User Input

[0328] input:

[0329] The terminal receives the prompt received from the server.

[0330] process:

[0331] The terminal displays a prompt, requesting input from the user.

[0332] output:

[0333] The information entered by the user (e.g., "Please tell us the reason for returning the product") is sent from the device to the emotion engine.

[0334] Step 9: Input analysis and emotion data generation by the emotion engine

[0335] input:

[0336] The emotion engine receives the user's input content from the terminal.

[0337] process:

[0338] The emotion engine uses a natural language processing API (e.g., Google Cloud NLP API) to analyze the input and generate emotion data. For example, it can extract emotions such as "anxiety" from the input.

[0339] output:

[0340] The generated emotion data is sent from the emotion engine to the server.

[0341] Step 10: Server Generates and Sends Response

[0342] input:

[0343] The server receives the emotion data sent from the emotion engine.

[0344] process:

[0345] The server generates a response for the customer based on the sentiment analysis results, such as "I'll explain why, but the product was damaged."

[0346] output:

[0347] The generated response is sent to the terminal.

[0348] Step 11: Displaying responses on the device and proceeding with the role-play

[0349] input:

[0350] The terminal receives the response from the server.

[0351] process:

[0352] The terminal displays the response content to the user, and also displays the progress of the simulation to the user.

[0353] output:

[0354] The user enters the next input and the roleplay continues.

[0355] Step 12: Feedback Generation by the Server

[0356] input:

[0357] User behavior history and emotion data when the scenario ends.

[0358] process:

[0359] The server analyzes the user's behavioral history and emotional data and generates feedback, evaluating the data based on various metrics using Python's Pandas library.

[0360] output:

[0361] The generated feedback is sent to the device in JSON format.

[0362] Step 13: Displaying feedback on the device

[0363] input:

[0364] The terminal receives the feedback sent from the server.

[0365] process:

[0366] The device displays the feedback to the user in a visually appealing format using HTML5 and CSS3.

[0367] output:

[0368] The user checks the feedback and uses it for the next simulation.

[0369] This series of processing steps allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. As a specific example, by selecting a "product return handling" scenario and role-playing, users can practically learn how to respond to customers' emotions. In addition, feedback provided after the system is completed provides users with specific guidelines for improving their own responses.

[0370] (Application example 2)

[0371] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0372] Conventional customer service skill learning systems have the problem that it is difficult for users to learn in an environment that closely resembles a real customer service situation. Furthermore, due to the lack of emotion recognition functionality, users are unable to learn how to respond appropriately to customer emotions. Furthermore, feedback is often simple, making it difficult for users to identify specific areas for improvement. The present invention aims to solve these problems and enable users to provide high-quality customer service in practice.

[0373] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for providing a role-play simulation for the user to learn customer service skills; means for the terminal to perform login authentication of the user; means for the server to verify user authentication; means for the terminal to display scenario options to the user; means for the server to prepare data based on the selected scenario; means for the terminal to display a screen for starting the scenario; means for the server to progress through the scenario and respond to input from the user; emotion engine means for the server to recognize the visitor's emotions in real time; means for the server to adjust the explanation speed and content according to the recognized emotions; means for the server to generate feedback after the scenario ends; and means for the terminal to display the feedback to the user. This enables the user to learn customer service skills that are in line with their emotions in real time and identify specific areas for improvement.

[0374] "Server" refers to a set of computer systems that provide data and services over a computer network.

[0375] "Terminal" refers to a device used by a user to access and operate a computer network.

[0376] "Login authentication" is a procedure for verifying a user's identity when accessing a system.

[0377] "Verifying user authentication" is the process of verifying whether the entered authentication information is correct by checking it against a database.

[0378] "Scenario options" refer to specific learning scenarios that a user can choose from during the learning process.

[0379] "Proceeding through a scenario" means executing a series of learning procedures based on a selected scenario.

[0380] "Emotion engine" refers to software or hardware for analyzing the emotions of users or visitors in real time.

[0381] "Feedback" is an evaluation or advice on improvement provided based on behavior and results after learning or work is completed.

[0382] The system for realizing this invention is composed of a server, a terminal, and an emotion engine. Each component operates as follows.

[0383] 1. Server initialization and login authentication

[0384] The server starts the system and initializes the basic components necessary to provide a role-play simulation environment for learning customer service skills. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters their authentication information, the server checks it against a database to verify that the user is a legitimate user. If authentication is successful, the scenario selection screen is displayed on the terminal.

[0385] 2. Scenario selection and preparation

[0386] The terminal displays a list of scenarios, and the user can select the one they are interested in. For example, scenarios such as "Handling Product Returns" or "Factory Tour Guide" can be selected. The server loads data corresponding to the selected scenario from the database, such as product information, guidance information, conversation templates, and simulation questions. Once the scenario is ready to start, that information is sent to the terminal.

[0387] 3. Role-play initiation and emotion recognition

[0388] The terminal displays a screen to start the scenario, and the server progresses the scenario. For example, in a "Factory Tour Guide" scenario, a prompt such as "We will move to the next tour station. Here, we will explain the manufacturing process" is generated and sent to the terminal. When the user inputs "Tell me more about the manufacturing process," the terminal sends the input to the emotion engine for analysis, which then sends it to the server. Based on the analysis results of the emotion engine, the server generates a response from the customer and sends it to the terminal. For example, if the user inputs something that sounds anxious, a response such as "Don't worry. Our manufacturing process is very safe" is returned.

[0389] 4. Providing Feedback

[0390] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user responded appropriately to the emotions and what areas need improvement. For example, specific advice such as "You conducted the tour appropriately, but next time, be sure to include details about the production line. To alleviate visitors' anxiety, we recommend explaining specific safety measures." The device displays this feedback to the user, who can then confirm it, improving both their customer service skills and their emotional understanding skills.

[0391] To implement this system, the following hardware and software are required:

[0392] Hardware:

[0393] Camera (for facial recognition of visitors)

[0394] Microphone (for voice input)

[0395] Speaker (for audio output)

[0396] A tablet or monitor (to display the interface)

[0397] Emotion recognition system (emotion engine)

[0398] software:

[0399] Python (control program)

[0400] OpenCV (image processing)

[0401] Google Cloud Speech-to-Text (voice recognition)

[0402] TENSORFLOW (registered trademark) (emotion recognition model)

[0403] Flask (web interface)

[0404] For example, if the prompt sentence "Generate the message to be displayed when a visitor shows a confused expression" is input into the generative AI model, the generated message will be "Please feel free to ask us if you have any questions. Let's explain the next station in more detail. Is that okay?" The feedback system can also be improved based on this prompt and message.

[0405] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0406] Step 1:

[0407] The server starts the system and initializes the basic components to provide a role-playing simulation environment for learning customer service skills. The inputs are system configuration information and user data, and the initialized system state is obtained as the output.

[0408] Step 2:

[0409] The terminal presents the user with a login screen and prompts them to enter their login ID and password, which provides the user's authentication information (login ID and password) as input and sends it to the server.

[0410] Step 3:

[0411] The server verifies whether the user is a legitimate user by checking the authentication information it receives against the database. The entered authentication information is compared with the information in the database, and if they match, authentication is successful and the output is that the user is a legitimate user.

[0412] Step 4:

[0413] A scenario selection screen is displayed on the terminal, and the user selects the scenario they are interested in. Here, the input is the user's selection, and the output is the selected scenario information.

[0414] Step 5:

[0415] The server loads relevant data from the database based on the selected scenario and prepares to start the scenario. The input is the selected scenario information, and the output is the scenario data (product information, conversation templates, etc.).

[0416] Step 6:

[0417] The terminal displays the scenario start screen, and the server progresses the scenario. It responds appropriately to input from the user. It provides an environment that is easy for the user to operate by displaying an interface such as prompts. The input is the scenario data and user operations, and the output is the progress of the scenario and the results of the user operations.

[0418] Step 7:

[0419] The server uses an emotion engine to recognize and analyze the user's emotions in real time. The input is the user's voice and video data, and the output is the recognized emotional information. An appropriate response is generated based on this emotional information.

[0420] Step 8:

[0421] The server adjusts the speed and content of the explanation according to the recognized emotion. For example, if the user is confused, the explanation will be slower and more detailed. The input is emotion information and scenario data, and the output is the adjusted explanation content.

[0422] Step 9:

[0423] When the scenario ends, the server generates feedback based on the user's behavioral history and emotional data. The input is the user's operation history and emotional data, and the output is specific feedback content.

[0424] Step 10:

[0425] The device generates feedback and displays it to the user, including specific areas for improvement and success that the user can use to improve their learning experience. The input is the feedback, and the output is the display to the user.

[0426] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0428] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0429] [Second embodiment]

[0430] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0431] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0432] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0433] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0434] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0436] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0437] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0438] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0439] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0440] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0441] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0442] The present invention relates to a system that provides a role-play simulation for users to learn customer service skills. This system is composed of a server and a terminal, each of which operates as follows.

[0443] 1. Server startup and login authentication

[0444] The server starts the system and allows the user to access it. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters their authentication information, the server checks it against a database to confirm that they are a legitimate user. If authentication is successful, the user proceeds to the scenario selection screen.

[0445] 2. Scenario selection and preparation

[0446] The terminal displays a list of scenarios, and the user selects the one they are interested in. For example, if they select the "Handling Product Returns" scenario, the server loads the corresponding data (product information, conversation templates, simulation questions, etc.) from the database, and the scenario is ready to begin.

[0447] 3. Begin the role-play

[0448] The terminal displays a screen to start the scenario, and the server runs the scenario. The server generates a prompt such as "A customer wishes to return an item. How would you like to proceed?" and sends it to the terminal. When the user inputs "We would like to know the reason for the return," the server analyzes the input, generates a response from the customer, and sends it to the terminal. For example, a response such as "We will explain the reason. The item was damaged" is returned.

[0449] 4. Providing Feedback

[0450] When the scenario ends, the server analyzes the user's overall response and generates feedback. The feedback includes what went well and what needs improvement. For example, the server might say, "We properly confirmed the reason for the return, but please remember to suggest a coupon next time." The terminal displays this feedback for the user to review.

[0451] This system allows users to efficiently improve their customer service skills by simulating actual customer service situations. In particular, the feedback function allows users to identify specific areas for improvement and reflect them in their actual work.

[0452] The processing flow will be explained below.

[0453] Step 1:

[0454] The server starts the system. The server initializes the basic components of the role-play simulation system and prepares to generate the login screen.

[0455] Step 2:

[0456] The terminal displays a login screen to the user. The terminal provides the user with a form to enter their login ID and password, and includes a "Login" button.

[0457] Step 3:

[0458] The user enters their login credentials. The user enters their login ID and password and clicks the "Login" button.

[0459] Step 4:

[0460] The server receives the user's authentication information and verifies the login ID and password against the database.

[0461] Step 5:

[0462] The server sends the authentication result to the terminal. If the authentication is successful, the server sends an instruction to the terminal to display the scenario selection screen.

[0463] Step 6:

[0464] The terminal displays a scenario selection screen. The terminal presents the user with a list of scenarios, offering options such as "handling product returns."

[0465] Step 7:

[0466] The user selects a scenario: the user clicks on the scenario that interests them.

[0467] Step 8:

[0468] The server loads data based on the selected scenario, and retrieves relevant product information, conversation templates, simulation questions, etc. from the database.

[0469] Step 9:

[0470] The server notifies the terminal that it is ready to start the scenario. The server sends a command to the terminal to start the scenario.

[0471] Step 10:

[0472] The terminal displays the scenario start screen. The terminal displays a screen that explains the scenario's outline and purpose to the user.

[0473] Step 11:

[0474] The server runs the scenario. It generates a prompt such as "The customer wants to return the product. How would you like to proceed?" and sends it to the terminal.

[0475] Step 12:

[0476] The user enters a response, such as "We would like to ask you the reason for the return," in the chat box and presses the send button.

[0477] Step 13:

[0478] The server processes the user's input. The server analyzes the user's input and generates a response depending on the progress of the scenario.

[0479] Step 14:

[0480] The server sends a response to the terminal, for example, "I'll explain why. The product was damaged."

[0481] Step 15:

[0482] The terminal displays the response to the user. The terminal displays the response received from the server in a chat box.

[0483] Step 16:

[0484] When the scenario ends, the server generates feedback by analyzing the user's response history and creating feedback that includes what went well and what needs to be improved.

[0485] Step 17:

[0486] The server sends the feedback to the device. The server sends the generated feedback to the device.

[0487] Step 18:

[0488] The terminal displays the feedback. The terminal visualizes the feedback sent from the server to the user.

[0489] Example 1

[0490] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0491] Conventional customer service skill learning systems have difficulty simulating actual customer service situations, making it difficult to effectively improve the skills required for real-world work. In particular, it is difficult to generate appropriate responses from a customer role in real time in response to user input and provide feedback based on that, which limits the effectiveness of learning.

[0492] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0493] In this invention, the server includes means for providing a role-playing simulation for a user to learn customer service skills, means for the server to analyze user input using a natural language processing library, means for the server to generate responses for the customer role using a generative AI model, and means for the server to generate feedback using an analytical model after the scenario ends. This allows users to effectively improve their customer service skills through a simulation that resembles a real-life customer service situation, and further allows them to receive specific feedback on their responses.

[0494] A "user" is a person who uses the system to learn customer service techniques.

[0495] "Customer service skills" refers to the skills and knowledge required to deal with customers.

[0496] "Role-play simulation" is a system that allows users to practice by simulating actual customer service situations.

[0497] A "server" is a computer device that performs the main processing of a system.

[0498] A "terminal" is a computer device or device that allows a user to operate the system.

[0499] "Login authentication" is the procedure for verifying a user's authentication information and confirming their access rights to the system.

[0500] "Authentication information" refers to information used when logging in, such as a user ID and password.

[0501] A "scenario" is a pre-set story that simulates a specific customer service situation.

[0502] "Scenario Choices" are options that allow the user to select the scenario they want to practice.

[0503] A "natural language processing library" is a program library for analyzing human language.

[0504] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input.

[0505] A "prompt" is text that prompts the user for the next action or response.

[0506] A "database" is a system for organizing, storing, and retrieving data.

[0507] "Feedback" refers to an evaluation of the user's response and areas for improvement.

[0508] An "analytical model" is an algorithm or program used to evaluate a user's performance.

[0509] The present invention relates to a system that provides role-play simulations for users to learn customer service skills. This system is comprised of a server and a terminal, and operates as follows.

[0510] Server startup and login authentication

[0511] The server starts the system and establishes a connection with the database. For example, a backend server built on a cloud service (e.g., a Node.js server on AWS) starts and connects to the MySQL database. The terminal displays a login screen to the user, and the user enters a login ID and password. For example, the user enters "user123" and "pass123." The server receives the entered authentication information and executes an SQL query to compare it with the database. If authentication is successful, the server returns the data of the scenario selection screen to the terminal, which displays it.

[0512] Scenario selection and preparation

[0513] The terminal displays a list of scenarios to the user. When the user selects the "Handling Product Returns" scenario, the selection information is sent to the server. The server loads data corresponding to the selected scenario (e.g., detailed product information, conversation templates, simulation questions, etc.) from the database. For example, the query "SELECT FROM scenarios WHERE scenario_name="Handling Product Returns";" is used for this purpose. The server caches the loaded data, and when it is ready, displays the scenario start screen on the terminal.

[0514] Beginning the role-play

[0515] The device displays a scenario start screen to the user and presents the initial prompt. In response, for example, the text "A customer wishes to return an item. How would you like to proceed?" is displayed. The user enters "I would like to know the reason for the return" and presses the send button. The server receives the input and analyzes it using a natural language processing library (e.g., spaCy or NLTK). A generative AI model (e.g., GPT-3) is used to generate a response for the customer, such as "I will explain the reason. The item was damaged." The generated response is then sent to the device, which displays it.

[0516] Providing Feedback

[0517] After the scenario is over, the server analyzes the user's response. For example, it analyzes the user's response log and uses a generative AI model to perform an overall evaluation. Specific feedback is generated, such as a message like, "We've properly confirmed the reason for the return, but please remember to suggest a coupon next time." The terminal displays the generated feedback to the user, who can then review it.

[0518] In this way, the system of the present invention allows users to simulate customer service situations in an environment that closely resembles a real customer service situation, effectively improving their customer service skills. In addition, by receiving specific feedback, users can improve their own customer service methods and reflect them in their actual work.

[0519] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0520] Step 1:

[0521] The server starts the system and establishes a connection to the database. The server starts a backend Node.js server and connects to the MySQL database. The system is now up and running and ready to accept user access.

[0522] Step 2:

[0523] The terminal displays a login screen and asks the user to enter authentication information. The user enters their login ID and password. Input: User's login ID and password. Output: User's authentication information is sent to the server.

[0524] Step 3:

[0525] The server checks the received authentication information against the database. The server executes an SQL query (e.g. SELECT FROM users WHERE user_id='user123' AND password='pass123';) to verify the authentication information. Input: User authentication information. Output: Authentication result (success or failure) is sent to the terminal.

[0526] Step 4:

[0527] The terminal displays the following screen to the user based on the authentication result. If authentication is successful, the terminal displays the scenario selection screen. Input: Authentication result. Output: Scenario selection screen.

[0528] Step 5:

[0529] The terminal displays a list of scenarios and lets the user select one. For example, the user selects the "Handling product returns" scenario. Input: User's scenario selection. Output: Selected scenario information is sent to the server.

[0530] Step 6:

[0531] The server loads data corresponding to the selected scenario from the database. For example, the server executes a query such as SELECT FROM scenarios WHERE scenario_name="Product return handling"; Input: Selected scenario information. Output: Corresponding scenario data is sent to the terminal.

[0532] Step 7:

[0533] The terminal displays the scenario start screen and presents the first prompt to the user. For example, the prompt may read, "A customer wishes to return an item. How would you like to proceed?" Input: Scenario data. Output: The first prompt is displayed to the user.

[0534] Step 8:

[0535] The user enters a response in the input box and submits it. For example, "Please tell us the reason for the return." Input: User's response. Output: User's response is sent to the server.

[0536] Step 9:

[0537] The server parses the user response using a natural language processing library, such as spaCy or NLTK. Input: User response. Output: Parsed data.

[0538] Step 10:

[0539] The server uses a generative AI model to generate a response for the customer. For example, it uses GPT-3 to generate a response like "I'll explain why. The item was damaged." Input: Parsed data. Output: Generated response for the customer.

[0540] Step 11:

[0541] The terminal displays the generated customer response to the user. Input: The generated customer response. Output: The customer response is displayed to the user.

[0542] Step 12:

[0543] After the scenario is completed, the server analyzes the user's responses and generates feedback. The analytical model is used to score the user's response log and identify areas for improvement. Input: User's response log. Output: Generated feedback.

[0544] Step 13:

[0545] The terminal displays the generated feedback to the user. For example, "We've properly verified the reason for the return, but don't forget to offer a coupon next time." Input: The generated feedback. Output: The feedback is displayed to the user.

[0546] This process allows users to simulate actual customer service situations and effectively learn customer service techniques. Through more specific responses and feedback, users can acquire skills that can be applied to actual work.

[0547] (Application example 1)

[0548] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0549] Conventional role-play simulation systems have limited methods for providing appropriate feedback to users learning customer service skills. As a result, users experience insufficient learning through the simulation, making it difficult to identify specific areas for improvement that can be applied in practice. Furthermore, while conventional systems have a general feedback function, they lack dynamic and personalized feedback based on input responses. This makes it difficult for users to learn the optimal response methods for each specific scenario. Furthermore, basic functions such as login authentication and scenario selection have also presented issues in terms of security and convenience.

[0550] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0551] In this invention, the server includes an information processing device that provides a role-playing simulation for a user to learn customer service skills; an operation terminal that performs user login authentication; an information processing device that verifies user authentication; an operation terminal that displays scenario options to the user; an information processing device that organizes data based on the selected scenario; an operation terminal that displays a screen indicating the start of the scenario; an information processing device that progresses through the scenario and responds to input from the user; an information processing device that generates feedback after the scenario ends; an operation terminal that displays the feedback to the user; a means for using a generative AI model to generate feedback based on user responses input from the operation terminal; and a means for inputting prompt sentences into the generative AI model. This allows the user to receive dynamic and personalized feedback in an interactive format and identify specific areas for improvement. This also provides high security and convenience, enabling effective and efficient improvement of customer service skills.

[0552] "User" refers to a person who uses the system to learn customer service skills, or the user of the system.

[0553] "Customer service skills" refer to the techniques and abilities required for dealing with customers, and are particularly required in customer service work in physical stores.

[0554] "Role-play simulation" refers to a simulation that simulates actual customer service work based on a specific scenario.

[0555] "Information processing device" refers to a computer device such as a server for processing and analyzing data, progressing a scenario, and the like.

[0556] An "operation terminal" is a device that is directly operated by a user, and includes mobile devices such as smartphones and tablets.

[0557] "Login authentication" refers to the process by which a user enters authentication information such as an ID and password to verify their identity in order to gain permission to access a system.

[0558] "User authentication" is the procedure in which the server verifies the authentication information entered by the user and confirms that the user is a legitimate user.

[0559] "Scenario options" refers to a list of multiple customer service scenarios that the user can experience, each of which simulates a specific situation.

[0560] "Preparing data" means preparing the information and settings required for the selected scenario and bringing it into a state ready for starting the simulation.

[0561] The "scenario start screen" refers to a screen that notifies the user of the start of a simulation.

[0562] A "generative AI model" is an artificial intelligence model that generates feedback and responses based on user input, and uses techniques such as machine learning.

[0563] A "prompt" is a question or instruction that tells the user what to do next.

[0564] The present invention relates to a system that provides role-play simulations for users to learn customer service skills. This system is composed of an information processing device (server) and an operation terminal (smartphone, tablet, etc.), and each operates as follows.

[0565] 1. User authentication:

[0566] The user accesses the system from an operation terminal and performs login authentication. When the user enters their ID and password, the information processing device verifies this and confirms that they are a legitimate user. If authentication is successful, the scenario selection screen is displayed.

[0567] 2. Scenario Selection:

[0568] The operation terminal displays multiple customer service scenarios (e.g., "handling product returns," "handling complaints," "product suggestions," etc.) to the user. When the user selects a scenario that interests them, the information processing device prepares the corresponding data (product information, conversation templates, simulation questions, etc.).

[0569] 3. Begin the role-play simulation:

[0570] The operating terminal displays a screen for starting the scenario, and interactive questions based on the scenario are displayed. The information processing device analyzes the user's input according to the progress of the scenario and generates an appropriate reply. For example, a prompt such as "A customer has complained about a defective product. How would you respond first?" is displayed. When the user enters a response, the information processing device analyzes it and generates the next response.

[0571] 4. Providing Feedback:

[0572] At the end of the scenario, the information processing device analyzes the user's overall response and generates feedback using a generative AI model. The feedback includes what was appropriate and what needs improvement. For example, specific advice such as "We properly confirmed the reason for the return, but don't forget to suggest a coupon next time" is provided. The operating device displays this feedback, allowing the user to confirm it.

[0573] Hardware and software used:

[0574] Hardware: smartphones, tablets, servers

[0575] Software: Python, API server, database, generative AI model

[0576] Examples:

[0577] The user opens the application and logs in by entering their ID and password. After logging in, they select the "Complaint Handling" scenario. The scenario begins and a prompt appears: "A customer has complained about a defective product. What is your first response?" The user enters their response and practices handling a complaint through a simulation. After the scenario ends, they review the provided feedback and improve their future responses.

[0578] In this way, users can efficiently improve their customer service skills by simulating actual customer service situations. Furthermore, interactive feedback allows users to identify specific areas for improvement and reflect them in their actual work.

[0579] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0580] Step 1:

[0581] (Login authentication)

[0582] The user enters an ID and password from the operating terminal. This is the input data. The terminal sends this authentication information to the server. The server verifies the authentication information against a database and confirms that the user is legitimate. If authentication is successful, the server generates a session token and sends it to the terminal. The terminal displays a scenario selection screen to the user. The output is the session token and the scenario selection screen.

[0583] Step 2:

[0584] (Scenario selection)

[0585] The user selects a scenario from the operating terminal. This is the input data. The terminal sends the selected scenario ID to the server. The server then prepares and retrieves the corresponding data (product information, conversation templates, simulation questions, etc.) from the database. The output is the prepared data.

[0586] Step 3:

[0587] (Scenario start screen display)

[0588] The terminal displays a scenario start screen to the user. A prompt sentence based on the scenario is displayed first. The input is the prepared data, and the output is the scenario start screen and the prompt sentence.

[0589] Step 4:

[0590] (Role-play simulation progress)

[0591] The user responds to the prompt sentence from the operating terminal. This response is the input data. The terminal sends this response to the server. The server uses a generative AI model to analyze the user's response and generate an appropriate reply. The generated reply is sent to the terminal and displayed to the user again. The output is the generated reply.

[0592] Step 5:

[0593] (Scenario end decision)

[0594] The server monitors the progress of the scenario and determines whether the scenario has finished. The progress of the scenario is the input data, and the decision on whether the scenario has finished is the output.

[0595] Step 6:

[0596] (Generating feedback)

[0597] Once the scenario is over, the server analyzes the user's overall response and generates feedback using a generative AI model, including what went well and what needs improvement. The input is the user's response log, and the output is the generated feedback.

[0598] Step 7:

[0599] (Show feedback)

[0600] The terminal displays the generated feedback to the user. The input is the generated feedback, and the output is the feedback display screen. The user can check the specific advice and improve their future actions.

[0601] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0602] The present invention relates to a system that provides a role-play simulation for users to learn customer service skills, and also combines it with an emotion engine that recognizes the user's emotions. This system is composed of a server means, a terminal means, and an emotion engine, each of which operates as follows.

[0603] 1. Server startup and login authentication

[0604] The server starts the system and initializes the basic components necessary to provide a learning environment for customer service skills. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters this authentication information, the server checks it against a database to confirm whether the user is a legitimate user. If authentication is successful, the scenario selection screen is displayed on the terminal.

[0605] 2. Scenario selection and preparation

[0606] The terminal displays a list of scenarios, and the user selects one that interests them, such as "Handling Product Returns." The server loads data corresponding to the selected scenario, such as product information, conversation templates, and simulation questions, from the database. Once the scenario is ready to start, that information is sent to the terminal.

[0607] 3. Role-play initiation and emotion recognition

[0608] The terminal displays a screen to start the scenario, and the server progresses the scenario. For example, a prompt such as "A customer wishes to return an item. How would you respond?" is generated and sent to the terminal. When the user inputs "I would like to know the reason for the return," the terminal sends the input to the emotion engine for analysis, which then sends the result to the server. Based on the analysis results of the emotion engine, the server generates a response for the customer and sends it to the terminal. For example, if the user inputs with an anxious look on their face, a response such as "I will explain the reason, but the item was damaged" is returned.

[0609] 4. Providing Feedback

[0610] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user handled the emotion appropriately and what areas need improvement. For example, specific advice such as "We properly confirmed the reason for the return, but next time, don't forget to offer a coupon. We recommend using more reassuring language to address the customer's concerns" may be provided. The device displays this feedback to the user, and by the user confirming it, the device can improve both customer service skills and emotional understanding skills.

[0611] This system allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. In particular, the addition of emotion recognition functionality allows users to learn how to respond to customers in accordance with their emotions, enabling them to provide high-quality service in real-world situations.

[0612] The processing flow will be explained below.

[0613] Step 1:

[0614] The server starts the system. The server initializes the basic components of the role-play simulation system and prepares to generate the login screen.

[0615] Step 2:

[0616] The terminal displays a login screen to the user. The terminal provides the user with a form to enter their login ID and password, and includes a "Login" button.

[0617] Step 3:

[0618] The user enters their login credentials. The user enters their login ID and password and clicks the "Login" button.

[0619] Step 4:

[0620] The server receives the user's authentication information and verifies the login ID and password against the database.

[0621] Step 5:

[0622] The server sends the authentication result to the terminal. If the authentication is successful, the server sends an instruction to the terminal to display the scenario selection screen.

[0623] Step 6:

[0624] The terminal displays a scenario selection screen. The terminal presents the user with a list of scenarios, offering options such as "handling product returns."

[0625] Step 7:

[0626] The user selects a scenario: the user clicks on the scenario that interests them.

[0627] Step 8:

[0628] The server loads data based on the selected scenario, and retrieves relevant product information, conversation templates, simulation questions, etc. from the database.

[0629] Step 9:

[0630] The server notifies the terminal that it is ready to start the scenario. The server sends a command to the terminal to start the scenario.

[0631] Step 10:

[0632] The terminal displays the scenario start screen. The terminal displays a screen that explains the scenario's outline and purpose to the user.

[0633] Step 11:

[0634] The server runs the scenario. It generates a prompt such as "The customer wants to return the product. How would you like to proceed?" and sends it to the terminal.

[0635] Step 12:

[0636] The user enters a response, such as "We would like to ask you the reason for the return," in the chat box and presses the send button.

[0637] Step 13:

[0638] The device sends the user's input to the emotion engine, which analyzes the input and tone of the words to identify the user's emotion.

[0639] Step 14:

[0640] The emotion engine sends the analysis results to the server, which include emotional information such as whether the user is anxious, angry, or relieved.

[0641] Step 15:

[0642] The server generates a response based on the user's emotional state: for example, if the user appears anxious, it generates a reassuring response such as, "The customer will explain why, but the item was damaged."

[0643] Step 16:

[0644] The server sends a response to the terminal. The generated response is sent to the terminal.

[0645] Step 17:

[0646] The terminal displays the response to the user. The terminal displays the response received from the server in a chat box.

[0647] Step 18:

[0648] When the scenario ends, the server generates feedback, summarizing the evaluation and areas for improvement based on the user's response history and the analysis results of the emotion engine.

[0649] Step 19:

[0650] The server sends the feedback to the terminal. The generated feedback is sent to the terminal.

[0651] Step 20:

[0652] The terminal displays the feedback. The terminal visualizes the feedback sent from the server to the user.

[0653] Example 2

[0654] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0655] In conventional customer service skill learning systems, it was difficult for users to learn appropriate responses based on actual customer emotional states. This resulted in problems such as insufficient acquisition of practical skills to improve on-site customer service skills and customer satisfaction. Furthermore, there was a lack of mechanisms to provide specific feedback on user behavior.

[0656] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0657] In this invention, the server includes computer means for providing a role-play simulation for a user to learn customer service skills, means for the terminal to perform login authentication of the user, means for the computer to verify user authentication, means for the terminal to display scenario options to the user, means for the computer to prepare data based on the selected scenario, means for the terminal to display a screen for starting the scenario, means for the computer to progress through the scenario and respond to input from the user, means for the terminal to send user input to an emotion engine, means for the emotion engine to analyze the user input and generate emotion data, means for the computer to generate a response based on the emotion analysis result, means for the computer to generate feedback after the scenario ends, and means for the terminal to display the feedback to the user. This enables the user to learn customer service skills tailored to the emotional states of actual customers and receive specific feedback.

[0658] "User" refers to a learner who uses the system to learn customer service skills.

[0659] "Computer means" refers to servers and cloud services that provide role-play simulations of customer service skills.

[0660] "Terminal" refers to a device such as a PC, tablet, or smartphone that a user uses to access the system.

[0661] "Login authentication" is the procedure for authenticating access to a system using a user's ID and password.

[0662] "User authentication verification" refers to the process of verifying whether the entered authentication information is legitimate.

[0663] "Scenario options" refers to multiple role-play scenarios that a user can choose from to learn.

[0664] "Data preparation" refers to a series of processes that load various information from a database based on a selected scenario.

[0665] The "scenario start screen" refers to the screen that is displayed to the user when the scenario starts.

[0666] "Scenario progression" refers to the process of managing the execution of the role-play simulation and the system's response to user input.

[0667] "Emotion engine" refers to software or algorithms that analyze user input and assess emotional state.

[0668] "Emotion data" refers to emotional information obtained by the emotion engine through analysis of user input.

[0669] "Response generation" refers to the process of creating an appropriate response to the user based on the analysis results of the emotion engine.

[0670] "Feedback" refers to information that provides an evaluation of the user's actions and decisions and specific advice after the scenario is completed.

[0671] "Displaying feedback" refers to presenting the generated feedback content to the user.

[0672] This invention relates to a system that provides a role-play simulation for users to learn customer service skills, and also combines it with an emotion engine that recognizes the user's emotions. The purpose of this system is to enable users to efficiently improve their customer service skills and emotional understanding in an environment that is similar to a real customer service situation. Specifically, the system is composed of a server, a terminal, and an emotion engine.

[0673] Server startup and login authentication

[0674] The server starts the system and initializes the basic components necessary to provide a learning environment for customer service skills. At this time, the server uses multiple components, including database connections, a user management system, and a scenario management system. Specifically, it uses a MySQL database, a Node.js server, and Nginx. The terminal provides the user with a login screen and prompts them to enter their login ID and password. When the user enters this authentication information, the server checks it against the database to confirm whether the user is a legitimate user. For example, it compares the hash values ​​of the user ID and password stored in the database. If authentication is successful, the scenario selection screen is displayed on the terminal.

[0675] Scenario selection and preparation

[0676] The device displays a list of scenarios, and the user selects the one they are interested in. For example, if the user selects a scenario such as "Handling Product Returns," the device sends the user's selection to the server. The server loads data corresponding to the selected scenario, such as product information, conversation templates, and simulation questions, from a database. At this time, the server efficiently retrieves the data using SQL queries, etc. Once the scenario is ready to start, that information is sent to the device. A framework such as React.js can be used by the device to display the scenario start screen.

[0677] Role-play initiation and emotion recognition

[0678] The device displays a screen to start the scenario, and the server runs the scenario. For example, a prompt such as "A customer wishes to return a product. How would you respond?" is generated and sent to the device. When generating this prompt, the server can use a generative AI model (e.g., OpenAI GPT-3). When the user enters "I would like to know the reason for the return," the device sends the input to an emotion engine for analysis. The emotion engine analyzes the input using Google Cloud's natural language processing API or similar and generates emotion data. For example, this emotion data may determine that the input expresses "anxiety." The analysis results are sent to the server, which generates a response based on the emotion analysis results and sends it to the device. For example, a response such as "I will explain the reason, but the product was damaged" may be returned.

[0679] Providing Feedback

[0680] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user appropriately responded to the user's emotions and what areas need improvement. For example, the server might provide specific advice such as, "We properly confirmed the reason for the return, but next time, please remember to offer a coupon. We recommend using more reassuring language to address the customer's concerns." The device displays this feedback to the user, and by allowing the user to review it, the server can improve both customer service skills and emotional understanding skills. When generating feedback, the server can use analytical tools such as Python's Pandas library. The device can also design the feedback screen to be visually appealing using HTML5 and CSS3.

[0681] This system allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. Specifically, by selecting a "product return handling" scenario and role-playing, users can practically learn how to respond to customer emotions. Furthermore, feedback provided after the system is completed provides users with specific guidelines for improving their own responses.

[0682] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0683] Step 1: Server startup and login authentication

[0684] input:

[0685] The server receives the boot command and starts booting the system.

[0686] process:

[0687] The server initializes the basic components (e.g., database connection, user management system, scenario management system) required to provide a learning environment for customer service skills.

[0688] Specific initialization operations include connecting to the MySQL database, starting the Node.js server, and configuring Nginx.

[0689] output:

[0690] The server logs the completion of initialization, generates a login screen, and sends it to the terminal.

[0691] Step 2: Provide login authentication via device

[0692] input:

[0693] The terminal receives the login screen information from the server.

[0694] process:

[0695] The terminal displays a login screen and prompts the user to enter their login ID and password.

[0696] output:

[0697] The user enters their login ID and password, and the device sends the authentication information to the server.

[0698] Step 3: Verifying User Authentication with the Server

[0699] input:

[0700] The server receives the authentication information sent from the terminal.

[0701] process:

[0702] The server checks the user information in the database to verify that the entered authentication information is valid. Specifically, it compares the user ID and hashed password with the information in the database.

[0703] output:

[0704] If authentication is successful, the server generates a scenario selection screen and sends it to the terminal. If authentication fails, it sends an error message to the terminal.

[0705] Step 4: Providing scenario selection on the device

[0706] input:

[0707] The terminal receives the scenario selection screen information from the server.

[0708] process:

[0709] The terminal displays a list of scenarios and prompts the user to select a scenario.

[0710] output:

[0711] The user selects a scenario of interest (e.g., "handling product returns"), and the terminal transmits the selection information to the server.

[0712] Step 5: Prepare scenario data on the server

[0713] input:

[0714] The server receives the scenario selection information transmitted from the terminal.

[0715] process:

[0716] The server loads data corresponding to the selected scenario (e.g., product information, conversation templates, simulation questions) from the database. Specifically, it retrieves the relevant data using an SQL query based on the scenario ID.

[0717] output:

[0718] The server transmits the prepared scenario data to the terminal.

[0719] Step 6: Providing scenario start by terminal

[0720] input:

[0721] The terminal receives the scenario data from the server.

[0722] process:

[0723] The device displays a scenario start screen. Specifically, the scenario start screen is visually generated using a framework such as React.js.

[0724] output:

[0725] Notify the user that the scenario is ready to begin.

[0726] Step 7: Start the role-play and generate prompts with the server

[0727] input:

[0728] The user confirms the start of the scenario.

[0729] process:

[0730] The server runs the scenario and generates key prompts (e.g., "A customer wants to return an item. How would you handle this?") using a generative AI model (e.g., OpenAI GPT-3).

[0731] output:

[0732] The generated prompt is sent to the terminal.

[0733] Step 8: Prompting the Terminal and Getting User Input

[0734] input:

[0735] The terminal receives the prompt received from the server.

[0736] process:

[0737] The terminal displays a prompt, requesting input from the user.

[0738] output:

[0739] The information entered by the user (e.g., "Please tell us the reason for returning the product") is sent from the device to the emotion engine.

[0740] Step 9: Input analysis and emotion data generation by the emotion engine

[0741] input:

[0742] The emotion engine receives the user's input content from the terminal.

[0743] process:

[0744] The emotion engine uses a natural language processing API (e.g., Google Cloud NLP API) to analyze the input and generate emotion data. For example, it can extract emotions such as "anxiety" from the input.

[0745] output:

[0746] The generated emotion data is sent from the emotion engine to the server.

[0747] Step 10: Server Generates and Sends Response

[0748] input:

[0749] The server receives the emotion data sent from the emotion engine.

[0750] process:

[0751] The server generates a response for the customer based on the sentiment analysis results, such as "I'll explain why, but the product was damaged."

[0752] output:

[0753] The generated response is sent to the terminal.

[0754] Step 11: Displaying responses on the device and proceeding with the role-play

[0755] input:

[0756] The terminal receives the response from the server.

[0757] process:

[0758] The terminal displays the response content to the user, and also displays the progress of the simulation to the user.

[0759] output:

[0760] The user enters the next input and the roleplay continues.

[0761] Step 12: Feedback Generation by the Server

[0762] input:

[0763] User behavior history and emotion data when the scenario ends.

[0764] process:

[0765] The server analyzes the user's behavioral history and emotional data and generates feedback, evaluating the data based on various metrics using Python's Pandas library.

[0766] output:

[0767] The generated feedback is sent to the device in JSON format.

[0768] Step 13: Displaying feedback on the device

[0769] input:

[0770] The terminal receives the feedback sent from the server.

[0771] process:

[0772] The device displays the feedback to the user in a visually appealing format using HTML5 and CSS3.

[0773] output:

[0774] The user checks the feedback and uses it for the next simulation.

[0775] This series of processing steps allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. As a specific example, by selecting a "product return handling" scenario and role-playing, users can practically learn how to respond to customers' emotions. In addition, feedback provided after the system is completed provides users with specific guidelines for improving their own responses.

[0776] (Application example 2)

[0777] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0778] Conventional customer service skill learning systems have the problem that it is difficult for users to learn in an environment that closely resembles a real customer service situation. Furthermore, due to the lack of emotion recognition functionality, users are unable to learn how to respond appropriately to customer emotions. Furthermore, feedback is often simple, making it difficult for users to identify specific areas for improvement. The present invention aims to solve these problems and enable users to provide high-quality customer service in practice.

[0779] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for providing a role-play simulation for the user to learn customer service skills; means for the terminal to perform login authentication of the user; means for the server to verify user authentication; means for the terminal to display scenario options to the user; means for the server to prepare data based on the selected scenario; means for the terminal to display a screen for starting the scenario; means for the server to progress through the scenario and respond to input from the user; emotion engine means for the server to recognize the visitor's emotions in real time; means for the server to adjust the explanation speed and content according to the recognized emotions; means for the server to generate feedback after the scenario ends; and means for the terminal to display the feedback to the user. This enables the user to learn customer service skills that are in line with their emotions in real time and identify specific areas for improvement.

[0780] "Server" refers to a set of computer systems that provide data and services over a computer network.

[0781] "Terminal" refers to a device used by a user to access and operate a computer network.

[0782] "Login authentication" is a procedure for verifying a user's identity when accessing a system.

[0783] "Verifying user authentication" is the process of verifying whether the entered authentication information is correct by checking it against a database.

[0784] "Scenario options" refer to specific learning scenarios that a user can choose from during the learning process.

[0785] "Proceeding through a scenario" means executing a series of learning procedures based on a selected scenario.

[0786] "Emotion engine" refers to software or hardware for analyzing the emotions of users or visitors in real time.

[0787] "Feedback" is an evaluation or advice on improvement provided based on behavior and results after learning or work is completed.

[0788] The system for realizing this invention is composed of a server, a terminal, and an emotion engine. Each component operates as follows.

[0789] 1. Server initialization and login authentication

[0790] The server starts the system and initializes the basic components necessary to provide a role-play simulation environment for learning customer service skills. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters their authentication information, the server checks it against a database to verify that the user is a legitimate user. If authentication is successful, the scenario selection screen is displayed on the terminal.

[0791] 2. Scenario selection and preparation

[0792] The terminal displays a list of scenarios, and the user can select the one they are interested in. For example, scenarios such as "Handling Product Returns" or "Factory Tour Guide" can be selected. The server loads data corresponding to the selected scenario from the database, such as product information, guidance information, conversation templates, and simulation questions. Once the scenario is ready to start, that information is sent to the terminal.

[0793] 3. Role-play initiation and emotion recognition

[0794] The terminal displays a screen to start the scenario, and the server progresses the scenario. For example, in a "Factory Tour Guide" scenario, a prompt such as "We will move to the next tour station. Here, we will explain the manufacturing process" is generated and sent to the terminal. When the user inputs "Tell me more about the manufacturing process," the terminal sends the input to the emotion engine for analysis, which then sends it to the server. Based on the analysis results of the emotion engine, the server generates a response from the customer and sends it to the terminal. For example, if the user inputs something that sounds anxious, a response such as "Don't worry. Our manufacturing process is very safe" is returned.

[0795] 4. Providing Feedback

[0796] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user responded appropriately to the emotions and what areas need improvement. For example, specific advice such as "You conducted the tour appropriately, but next time, be sure to include details about the production line. To alleviate visitors' anxiety, we recommend explaining specific safety measures." The device displays this feedback to the user, who can then confirm it, improving both their customer service skills and their emotional understanding skills.

[0797] To implement this system, the following hardware and software are required:

[0798] Hardware:

[0799] Camera (for facial recognition of visitors)

[0800] Microphone (for voice input)

[0801] Speaker (for audio output)

[0802] A tablet or monitor (to display the interface)

[0803] Emotion recognition system (emotion engine)

[0804] software:

[0805] Python (control program)

[0806] OpenCV (image processing)

[0807] Google Cloud Speech-to-Text (voice recognition)

[0808] TensorFlow (emotion recognition model)

[0809] Flask (web interface)

[0810] For example, if the prompt sentence "Generate the message to be displayed when a visitor shows a confused expression" is input into the generative AI model, the generated message will be "Please feel free to ask us if you have any questions. Let's explain the next station in more detail. Is that okay?" The feedback system can also be improved based on this prompt and message.

[0811] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0812] Step 1:

[0813] The server starts the system and initializes the basic components to provide a role-playing simulation environment for learning customer service skills. The inputs are system configuration information and user data, and the initialized system state is obtained as the output.

[0814] Step 2:

[0815] The terminal presents the user with a login screen and prompts them to enter their login ID and password, which provides the user's authentication information (login ID and password) as input and sends it to the server.

[0816] Step 3:

[0817] The server verifies whether the user is a legitimate user by checking the authentication information it receives against the database. The entered authentication information is compared with the information in the database, and if they match, authentication is successful and the output is that the user is a legitimate user.

[0818] Step 4:

[0819] A scenario selection screen is displayed on the terminal, and the user selects the scenario they are interested in. Here, the input is the user's selection, and the output is the selected scenario information.

[0820] Step 5:

[0821] The server loads relevant data from the database based on the selected scenario and prepares to start the scenario. The input is the selected scenario information, and the output is the scenario data (product information, conversation templates, etc.).

[0822] Step 6:

[0823] The terminal displays the scenario start screen, and the server progresses the scenario. It responds appropriately to input from the user. It provides an environment that is easy for the user to operate by displaying an interface such as prompts. The input is the scenario data and user operations, and the output is the progress of the scenario and the results of the user operations.

[0824] Step 7:

[0825] The server uses an emotion engine to recognize and analyze the user's emotions in real time. The input is the user's voice and video data, and the output is the recognized emotional information. An appropriate response is generated based on this emotional information.

[0826] Step 8:

[0827] The server adjusts the speed and content of the explanation according to the recognized emotion. For example, if the user is confused, the explanation will be slower and more detailed. The input is emotion information and scenario data, and the output is the adjusted explanation content.

[0828] Step 9:

[0829] When the scenario ends, the server generates feedback based on the user's behavioral history and emotional data. The input is the user's operation history and emotional data, and the output is specific feedback content.

[0830] Step 10:

[0831] The device generates feedback and displays it to the user, including specific areas for improvement and success that the user can use to improve their learning experience. The input is the feedback, and the output is the display to the user.

[0832] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0833] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0834] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0835] [Third embodiment]

[0836] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0837] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0838] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0839] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0840] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0841] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0842] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0843] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0844] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0845] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0846] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0847] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0848] The present invention relates to a system that provides a role-play simulation for users to learn customer service skills. This system is composed of a server and a terminal, each of which operates as follows.

[0849] 1. Server startup and login authentication

[0850] The server starts the system and allows the user to access it. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters their authentication information, the server checks it against a database to confirm that they are a legitimate user. If authentication is successful, the user proceeds to the scenario selection screen.

[0851] 2. Scenario selection and preparation

[0852] The terminal displays a list of scenarios, and the user selects the one they are interested in. For example, if they select the "Handling Product Returns" scenario, the server loads the corresponding data (product information, conversation templates, simulation questions, etc.) from the database, and the scenario is ready to begin.

[0853] 3. Begin the role-play

[0854] The terminal displays a screen to start the scenario, and the server runs the scenario. The server generates a prompt such as "A customer wishes to return an item. How would you like to proceed?" and sends it to the terminal. When the user inputs "We would like to know the reason for the return," the server analyzes the input, generates a response from the customer, and sends it to the terminal. For example, a response such as "We will explain the reason. The item was damaged" is returned.

[0855] 4. Providing Feedback

[0856] When the scenario ends, the server analyzes the user's overall response and generates feedback. The feedback includes what went well and what needs improvement. For example, the server might say, "We properly confirmed the reason for the return, but please remember to suggest a coupon next time." The terminal displays this feedback for the user to review.

[0857] This system allows users to efficiently improve their customer service skills by simulating actual customer service situations. In particular, the feedback function allows users to identify specific areas for improvement and reflect them in their actual work.

[0858] The processing flow will be explained below.

[0859] Step 1:

[0860] The server starts the system. The server initializes the basic components of the role-play simulation system and prepares to generate the login screen.

[0861] Step 2:

[0862] The terminal displays a login screen to the user. The terminal provides the user with a form to enter their login ID and password, and includes a "Login" button.

[0863] Step 3:

[0864] The user enters their login credentials. The user enters their login ID and password and clicks the "Login" button.

[0865] Step 4:

[0866] The server receives the user's authentication information and verifies the login ID and password against the database.

[0867] Step 5:

[0868] The server sends the authentication result to the terminal. If the authentication is successful, the server sends an instruction to the terminal to display the scenario selection screen.

[0869] Step 6:

[0870] The terminal displays a scenario selection screen. The terminal presents the user with a list of scenarios, offering options such as "handling product returns."

[0871] Step 7:

[0872] The user selects a scenario: the user clicks on the scenario that interests them.

[0873] Step 8:

[0874] The server loads data based on the selected scenario, and retrieves relevant product information, conversation templates, simulation questions, etc. from the database.

[0875] Step 9:

[0876] The server notifies the terminal that it is ready to start the scenario. The server sends a command to the terminal to start the scenario.

[0877] Step 10:

[0878] The terminal displays the scenario start screen. The terminal displays a screen that explains the scenario's outline and purpose to the user.

[0879] Step 11:

[0880] The server runs the scenario. It generates a prompt such as "The customer wants to return the product. How would you like to proceed?" and sends it to the terminal.

[0881] Step 12:

[0882] The user enters a response, such as "We would like to ask you the reason for the return," in the chat box and presses the send button.

[0883] Step 13:

[0884] The server processes the user's input. The server analyzes the user's input and generates a response depending on the progress of the scenario.

[0885] Step 14:

[0886] The server sends a response to the terminal, for example, "I'll explain why. The product was damaged."

[0887] Step 15:

[0888] The terminal displays the response to the user. The terminal displays the response received from the server in a chat box.

[0889] Step 16:

[0890] When the scenario ends, the server generates feedback by analyzing the user's response history and creating feedback that includes what went well and what needs to be improved.

[0891] Step 17:

[0892] The server sends the feedback to the device. The server sends the generated feedback to the device.

[0893] Step 18:

[0894] The terminal displays the feedback. The terminal visualizes the feedback sent from the server to the user.

[0895] Example 1

[0896] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0897] Conventional customer service skill learning systems have difficulty simulating actual customer service situations, making it difficult to effectively improve the skills required for real-world work. In particular, it is difficult to generate appropriate responses from a customer role in real time in response to user input and provide feedback based on that, which limits the effectiveness of learning.

[0898] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0899] In this invention, the server includes means for providing a role-playing simulation for a user to learn customer service skills, means for the server to analyze user input using a natural language processing library, means for the server to generate responses for the customer role using a generative AI model, and means for the server to generate feedback using an analytical model after the scenario ends. This allows users to effectively improve their customer service skills through a simulation that resembles a real-life customer service situation, and further allows them to receive specific feedback on their responses.

[0900] A "user" is a person who uses the system to learn customer service techniques.

[0901] "Customer service skills" refers to the skills and knowledge required to deal with customers.

[0902] "Role-play simulation" is a system that allows users to practice by simulating actual customer service situations.

[0903] A "server" is a computer device that performs the main processing of a system.

[0904] A "terminal" is a computer device or device that allows a user to operate the system.

[0905] "Login authentication" is the procedure for verifying a user's authentication information and confirming their access rights to the system.

[0906] "Authentication information" refers to information used when logging in, such as a user ID and password.

[0907] A "scenario" is a pre-set story that simulates a specific customer service situation.

[0908] "Scenario Choices" are options that allow the user to select the scenario they want to practice.

[0909] A "natural language processing library" is a program library for analyzing human language.

[0910] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input.

[0911] A "prompt" is text that prompts the user for the next action or response.

[0912] A "database" is a system for organizing, storing, and retrieving data.

[0913] "Feedback" refers to an evaluation of the user's response and areas for improvement.

[0914] An "analytical model" is an algorithm or program used to evaluate a user's performance.

[0915] The present invention relates to a system that provides role-play simulations for users to learn customer service skills. This system is comprised of a server and a terminal, and operates as follows.

[0916] Server startup and login authentication

[0917] The server starts the system and establishes a connection with the database. For example, a backend server built on a cloud service (e.g., a Node.js server on AWS) starts and connects to the MySQL database. The terminal displays a login screen to the user, and the user enters a login ID and password. For example, the user enters "user123" and "pass123." The server receives the entered authentication information and executes an SQL query to compare it with the database. If authentication is successful, the server returns the data of the scenario selection screen to the terminal, which displays it.

[0918] Scenario selection and preparation

[0919] The terminal displays a list of scenarios to the user. When the user selects the "Handling Product Returns" scenario, the selection information is sent to the server. The server loads data corresponding to the selected scenario (e.g., detailed product information, conversation templates, simulation questions, etc.) from the database. For example, the query "SELECT FROM scenarios WHERE scenario_name="Handling Product Returns";" is used for this purpose. The server caches the loaded data, and when it is ready, displays the scenario start screen on the terminal.

[0920] Beginning the role-play

[0921] The device displays a scenario start screen to the user and presents the initial prompt. In response, for example, the text "A customer wishes to return an item. How would you like to proceed?" is displayed. The user enters "I would like to know the reason for the return" and presses the send button. The server receives the input and analyzes it using a natural language processing library (e.g., spaCy or NLTK). A generative AI model (e.g., GPT-3) is used to generate a response for the customer, such as "I will explain the reason. The item was damaged." The generated response is then sent to the device, which displays it.

[0922] Providing Feedback

[0923] After the scenario is over, the server analyzes the user's response. For example, it analyzes the user's response log and uses a generative AI model to perform an overall evaluation. Specific feedback is generated, such as a message like, "We've properly confirmed the reason for the return, but please remember to suggest a coupon next time." The terminal displays the generated feedback to the user, who can then review it.

[0924] In this way, the system of the present invention allows users to simulate customer service situations in an environment that closely resembles a real customer service situation, effectively improving their customer service skills. In addition, by receiving specific feedback, users can improve their own customer service methods and reflect them in their actual work.

[0925] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0926] Step 1:

[0927] The server starts the system and establishes a connection to the database. The server starts a backend Node.js server and connects to the MySQL database. The system is now up and running and ready to accept user access.

[0928] Step 2:

[0929] The terminal displays a login screen and asks the user to enter authentication information. The user enters their login ID and password. Input: User's login ID and password. Output: User's authentication information is sent to the server.

[0930] Step 3:

[0931] The server checks the received authentication information against the database. The server executes an SQL query (e.g. SELECT FROM users WHERE user_id='user123' AND password='pass123';) to verify the authentication information. Input: User authentication information. Output: Authentication result (success or failure) is sent to the terminal.

[0932] Step 4:

[0933] The terminal displays the following screen to the user based on the authentication result. If authentication is successful, the terminal displays the scenario selection screen. Input: Authentication result. Output: Scenario selection screen.

[0934] Step 5:

[0935] The terminal displays a list of scenarios and lets the user select one. For example, the user selects the "Handling product returns" scenario. Input: User's scenario selection. Output: Selected scenario information is sent to the server.

[0936] Step 6:

[0937] The server loads data corresponding to the selected scenario from the database. For example, the server executes a query such as SELECT FROM scenarios WHERE scenario_name="Product return handling"; Input: Selected scenario information. Output: Corresponding scenario data is sent to the terminal.

[0938] Step 7:

[0939] The terminal displays the scenario start screen and presents the first prompt to the user. For example, the prompt may read, "A customer wishes to return an item. How would you like to proceed?" Input: Scenario data. Output: The first prompt is displayed to the user.

[0940] Step 8:

[0941] The user enters a response in the input box and submits it. For example, "Please tell us the reason for the return." Input: User's response. Output: User's response is sent to the server.

[0942] Step 9:

[0943] The server parses the user response using a natural language processing library, such as spaCy or NLTK. Input: User response. Output: Parsed data.

[0944] Step 10:

[0945] The server uses a generative AI model to generate a response for the customer. For example, it uses GPT-3 to generate a response like "I'll explain why. The item was damaged." Input: Parsed data. Output: Generated response for the customer.

[0946] Step 11:

[0947] The terminal displays the generated customer response to the user. Input: The generated customer response. Output: The customer response is displayed to the user.

[0948] Step 12:

[0949] After the scenario is completed, the server analyzes the user's responses and generates feedback. The analytical model is used to score the user's response log and identify areas for improvement. Input: User's response log. Output: Generated feedback.

[0950] Step 13:

[0951] The terminal displays the generated feedback to the user. For example, "We've properly verified the reason for the return, but don't forget to offer a coupon next time." Input: The generated feedback. Output: The feedback is displayed to the user.

[0952] This process allows users to simulate actual customer service situations and effectively learn customer service techniques. Through more specific responses and feedback, users can acquire skills that can be applied to actual work.

[0953] (Application example 1)

[0954] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0955] Conventional role-play simulation systems have limited methods for providing appropriate feedback to users learning customer service skills. As a result, users experience insufficient learning through the simulation, making it difficult to identify specific areas for improvement that can be applied in practice. Furthermore, while conventional systems have a general feedback function, they lack dynamic and personalized feedback based on input responses. This makes it difficult for users to learn the optimal response methods for each specific scenario. Furthermore, basic functions such as login authentication and scenario selection have also presented issues in terms of security and convenience.

[0956] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0957] In this invention, the server includes an information processing device that provides a role-playing simulation for a user to learn customer service skills; an operation terminal that performs user login authentication; an information processing device that verifies user authentication; an operation terminal that displays scenario options to the user; an information processing device that organizes data based on the selected scenario; an operation terminal that displays a screen indicating the start of the scenario; an information processing device that progresses through the scenario and responds to input from the user; an information processing device that generates feedback after the scenario ends; an operation terminal that displays the feedback to the user; a means for using a generative AI model to generate feedback based on user responses input from the operation terminal; and a means for inputting prompt sentences into the generative AI model. This allows the user to receive dynamic and personalized feedback in an interactive format and identify specific areas for improvement. This also provides high security and convenience, enabling effective and efficient improvement of customer service skills.

[0958] "User" refers to a person who uses the system to learn customer service skills, or the user of the system.

[0959] "Customer service skills" refer to the techniques and abilities required for dealing with customers, and are particularly required in customer service work in physical stores.

[0960] "Role-play simulation" refers to a simulation that simulates actual customer service work based on a specific scenario.

[0961] "Information processing device" refers to a computer device such as a server for processing and analyzing data, progressing a scenario, and the like.

[0962] An "operation terminal" is a device that is directly operated by a user, and includes mobile devices such as smartphones and tablets.

[0963] "Login authentication" refers to the process by which a user enters authentication information such as an ID and password to verify their identity in order to gain permission to access a system.

[0964] "User authentication" is the procedure in which the server verifies the authentication information entered by the user and confirms that the user is a legitimate user.

[0965] "Scenario options" refers to a list of multiple customer service scenarios that the user can experience, each of which simulates a specific situation.

[0966] "Preparing data" means preparing the information and settings required for the selected scenario and bringing it into a state ready for starting the simulation.

[0967] The "scenario start screen" refers to a screen that notifies the user of the start of a simulation.

[0968] A "generative AI model" is an artificial intelligence model that generates feedback and responses based on user input, and uses techniques such as machine learning.

[0969] A "prompt" is a question or instruction that tells the user what to do next.

[0970] The present invention relates to a system that provides role-play simulations for users to learn customer service skills. This system is composed of an information processing device (server) and an operation terminal (smartphone, tablet, etc.), and each operates as follows.

[0971] 1. User authentication:

[0972] The user accesses the system from an operation terminal and performs login authentication. When the user enters their ID and password, the information processing device verifies this and confirms that they are a legitimate user. If authentication is successful, the scenario selection screen is displayed.

[0973] 2. Scenario Selection:

[0974] The operation terminal displays multiple customer service scenarios (e.g., "handling product returns," "handling complaints," "product suggestions," etc.) to the user. When the user selects a scenario that interests them, the information processing device prepares the corresponding data (product information, conversation templates, simulation questions, etc.).

[0975] 3. Begin the role-play simulation:

[0976] The operating terminal displays a screen for starting the scenario, and interactive questions based on the scenario are displayed. The information processing device analyzes the user's input according to the progress of the scenario and generates an appropriate reply. For example, a prompt such as "A customer has complained about a defective product. How would you respond first?" is displayed. When the user enters a response, the information processing device analyzes it and generates the next response.

[0977] 4. Providing Feedback:

[0978] At the end of the scenario, the information processing device analyzes the user's overall response and generates feedback using a generative AI model. The feedback includes what was appropriate and what needs improvement. For example, specific advice such as "We properly confirmed the reason for the return, but don't forget to suggest a coupon next time" is provided. The operating device displays this feedback, allowing the user to confirm it.

[0979] Hardware and software used:

[0980] Hardware: smartphones, tablets, servers

[0981] Software: Python, API server, database, generative AI model

[0982] Examples:

[0983] The user opens the application and logs in by entering their ID and password. After logging in, they select the "Complaint Handling" scenario. The scenario begins and a prompt appears: "A customer has complained about a defective product. What is your first response?" The user enters their response and practices handling a complaint through a simulation. After the scenario ends, they review the provided feedback and improve their future responses.

[0984] In this way, users can efficiently improve their customer service skills by simulating actual customer service situations. Furthermore, interactive feedback allows users to identify specific areas for improvement and reflect them in their actual work.

[0985] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0986] Step 1:

[0987] (Login authentication)

[0988] The user enters an ID and password from the operating terminal. This is the input data. The terminal sends this authentication information to the server. The server verifies the authentication information against a database and confirms that the user is legitimate. If authentication is successful, the server generates a session token and sends it to the terminal. The terminal displays a scenario selection screen to the user. The output is the session token and the scenario selection screen.

[0989] Step 2:

[0990] (Scenario selection)

[0991] The user selects a scenario from the operating terminal. This is the input data. The terminal sends the selected scenario ID to the server. The server then prepares and retrieves the corresponding data (product information, conversation templates, simulation questions, etc.) from the database. The output is the prepared data.

[0992] Step 3:

[0993] (Scenario start screen display)

[0994] The terminal displays a scenario start screen to the user. A prompt sentence based on the scenario is displayed first. The input is the prepared data, and the output is the scenario start screen and the prompt sentence.

[0995] Step 4:

[0996] (Role-play simulation progress)

[0997] The user responds to the prompt sentence from the operating terminal. This response is the input data. The terminal sends this response to the server. The server uses a generative AI model to analyze the user's response and generate an appropriate reply. The generated reply is sent to the terminal and displayed to the user again. The output is the generated reply.

[0998] Step 5:

[0999] (Scenario end decision)

[1000] The server monitors the progress of the scenario and determines whether the scenario has finished. The progress of the scenario is the input data, and the decision on whether the scenario has finished is the output.

[1001] Step 6:

[1002] (Generating feedback)

[1003] Once the scenario is over, the server analyzes the user's overall response and generates feedback using a generative AI model, including what went well and what needs improvement. The input is the user's response log, and the output is the generated feedback.

[1004] Step 7:

[1005] (Show feedback)

[1006] The terminal displays the generated feedback to the user. The input is the generated feedback, and the output is the feedback display screen. The user can check the specific advice and improve their future actions.

[1007] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1008] The present invention relates to a system that provides a role-play simulation for users to learn customer service skills, and also combines it with an emotion engine that recognizes the user's emotions. This system is composed of a server means, a terminal means, and an emotion engine, each of which operates as follows.

[1009] 1. Server startup and login authentication

[1010] The server starts the system and initializes the basic components necessary to provide a learning environment for customer service skills. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters this authentication information, the server checks it against a database to confirm whether the user is a legitimate user. If authentication is successful, the scenario selection screen is displayed on the terminal.

[1011] 2. Scenario selection and preparation

[1012] The terminal displays a list of scenarios, and the user selects one that interests them, such as "Handling Product Returns." The server loads data corresponding to the selected scenario, such as product information, conversation templates, and simulation questions, from the database. Once the scenario is ready to start, that information is sent to the terminal.

[1013] 3. Role-play initiation and emotion recognition

[1014] The terminal displays a screen to start the scenario, and the server progresses the scenario. For example, a prompt such as "A customer wishes to return an item. How would you respond?" is generated and sent to the terminal. When the user inputs "I would like to know the reason for the return," the terminal sends the input to the emotion engine for analysis, which then sends the result to the server. Based on the analysis results of the emotion engine, the server generates a response for the customer and sends it to the terminal. For example, if the user inputs with an anxious look on their face, a response such as "I will explain the reason, but the item was damaged" is returned.

[1015] 4. Providing Feedback

[1016] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user handled the emotion appropriately and what areas need improvement. For example, specific advice such as "We properly confirmed the reason for the return, but next time, don't forget to offer a coupon. We recommend using more reassuring language to address the customer's concerns" may be provided. The device displays this feedback to the user, and by the user confirming it, the device can improve both customer service skills and emotional understanding skills.

[1017] This system allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. In particular, the addition of emotion recognition functionality allows users to learn how to respond to customers in accordance with their emotions, enabling them to provide high-quality service in real-world situations.

[1018] The processing flow will be explained below.

[1019] Step 1:

[1020] The server starts the system. The server initializes the basic components of the role-play simulation system and prepares to generate the login screen.

[1021] Step 2:

[1022] The terminal displays a login screen to the user. The terminal provides the user with a form to enter their login ID and password, and includes a "Login" button.

[1023] Step 3:

[1024] The user enters their login credentials. The user enters their login ID and password and clicks the "Login" button.

[1025] Step 4:

[1026] The server receives the user's authentication information and verifies the login ID and password against the database.

[1027] Step 5:

[1028] The server sends the authentication result to the terminal. If the authentication is successful, the server sends an instruction to the terminal to display the scenario selection screen.

[1029] Step 6:

[1030] The terminal displays a scenario selection screen. The terminal presents the user with a list of scenarios, offering options such as "handling product returns."

[1031] Step 7:

[1032] The user selects a scenario: the user clicks on the scenario that interests them.

[1033] Step 8:

[1034] The server loads data based on the selected scenario, and retrieves relevant product information, conversation templates, simulation questions, etc. from the database.

[1035] Step 9:

[1036] The server notifies the terminal that it is ready to start the scenario. The server sends a command to the terminal to start the scenario.

[1037] Step 10:

[1038] The terminal displays the scenario start screen. The terminal displays a screen that explains the scenario's outline and purpose to the user.

[1039] Step 11:

[1040] The server runs the scenario. It generates a prompt such as "The customer wants to return the product. How would you like to proceed?" and sends it to the terminal.

[1041] Step 12:

[1042] The user enters a response, such as "We would like to ask you the reason for the return," in the chat box and presses the send button.

[1043] Step 13:

[1044] The device sends the user's input to the emotion engine, which analyzes the input and tone of the words to identify the user's emotion.

[1045] Step 14:

[1046] The emotion engine sends the analysis results to the server, which include emotional information such as whether the user is anxious, angry, or relieved.

[1047] Step 15:

[1048] The server generates a response based on the user's emotional state: for example, if the user appears anxious, it generates a reassuring response such as, "The customer will explain why, but the item was damaged."

[1049] Step 16:

[1050] The server sends a response to the terminal. The generated response is sent to the terminal.

[1051] Step 17:

[1052] The terminal displays the response to the user. The terminal displays the response received from the server in a chat box.

[1053] Step 18:

[1054] When the scenario ends, the server generates feedback, summarizing the evaluation and areas for improvement based on the user's response history and the analysis results of the emotion engine.

[1055] Step 19:

[1056] The server sends the feedback to the terminal. The generated feedback is sent to the terminal.

[1057] Step 20:

[1058] The terminal displays the feedback. The terminal visualizes the feedback sent from the server to the user.

[1059] Example 2

[1060] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1061] In conventional customer service skill learning systems, it was difficult for users to learn appropriate responses based on actual customer emotional states. This resulted in problems such as insufficient acquisition of practical skills to improve on-site customer service skills and customer satisfaction. Furthermore, there was a lack of mechanisms to provide specific feedback on user behavior.

[1062] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1063] In this invention, the server includes computer means for providing a role-play simulation for a user to learn customer service skills, means for the terminal to perform login authentication of the user, means for the computer to verify user authentication, means for the terminal to display scenario options to the user, means for the computer to prepare data based on the selected scenario, means for the terminal to display a screen for starting the scenario, means for the computer to progress through the scenario and respond to input from the user, means for the terminal to send user input to an emotion engine, means for the emotion engine to analyze the user input and generate emotion data, means for the computer to generate a response based on the emotion analysis result, means for the computer to generate feedback after the scenario ends, and means for the terminal to display the feedback to the user. This enables the user to learn customer service skills tailored to the emotional states of actual customers and receive specific feedback.

[1064] "User" refers to a learner who uses the system to learn customer service skills.

[1065] "Computer means" refers to servers and cloud services that provide role-play simulations of customer service skills.

[1066] "Terminal" refers to a device such as a PC, tablet, or smartphone that a user uses to access the system.

[1067] "Login authentication" is the procedure for authenticating access to a system using a user's ID and password.

[1068] "User authentication verification" refers to the process of verifying whether the entered authentication information is legitimate.

[1069] "Scenario options" refers to multiple role-play scenarios that a user can choose from to learn.

[1070] "Data preparation" refers to a series of processes that load various information from a database based on a selected scenario.

[1071] The "scenario start screen" refers to the screen that is displayed to the user when the scenario starts.

[1072] "Scenario progression" refers to the process of managing the execution of the role-play simulation and the system's response to user input.

[1073] "Emotion engine" refers to software or algorithms that analyze user input and assess emotional state.

[1074] "Emotion data" refers to emotional information obtained by the emotion engine through analysis of user input.

[1075] "Response generation" refers to the process of creating an appropriate response to the user based on the analysis results of the emotion engine.

[1076] "Feedback" refers to information that provides an evaluation of the user's actions and decisions and specific advice after the scenario is completed.

[1077] "Displaying feedback" refers to presenting the generated feedback content to the user.

[1078] This invention relates to a system that provides a role-play simulation for users to learn customer service skills, and also combines it with an emotion engine that recognizes the user's emotions. The purpose of this system is to enable users to efficiently improve their customer service skills and emotional understanding in an environment that is similar to a real customer service situation. Specifically, the system is composed of a server, a terminal, and an emotion engine.

[1079] Server startup and login authentication

[1080] The server starts the system and initializes the basic components necessary to provide a learning environment for customer service skills. At this time, the server uses multiple components, including database connections, a user management system, and a scenario management system. Specifically, it uses a MySQL database, a Node.js server, and Nginx. The terminal provides the user with a login screen and prompts them to enter their login ID and password. When the user enters this authentication information, the server checks it against the database to confirm whether the user is a legitimate user. For example, it compares the hash values ​​of the user ID and password stored in the database. If authentication is successful, the scenario selection screen is displayed on the terminal.

[1081] Scenario selection and preparation

[1082] The device displays a list of scenarios, and the user selects the one they are interested in. For example, if the user selects a scenario such as "Handling Product Returns," the device sends the user's selection to the server. The server loads data corresponding to the selected scenario, such as product information, conversation templates, and simulation questions, from a database. At this time, the server efficiently retrieves the data using SQL queries, etc. Once the scenario is ready to start, that information is sent to the device. A framework such as React.js can be used by the device to display the scenario start screen.

[1083] Role-play initiation and emotion recognition

[1084] The device displays a screen to start the scenario, and the server runs the scenario. For example, a prompt such as "A customer wishes to return a product. How would you respond?" is generated and sent to the device. When generating this prompt, the server can use a generative AI model (e.g., OpenAI GPT-3). When the user enters "I would like to know the reason for the return," the device sends the input to an emotion engine for analysis. The emotion engine analyzes the input using Google Cloud's natural language processing API or similar and generates emotion data. For example, this emotion data may determine that the input expresses "anxiety." The analysis results are sent to the server, which generates a response based on the emotion analysis results and sends it to the device. For example, a response such as "I will explain the reason, but the product was damaged" may be returned.

[1085] Providing Feedback

[1086] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user appropriately responded to the user's emotions and what areas need improvement. For example, the server might provide specific advice such as, "We properly confirmed the reason for the return, but next time, please remember to offer a coupon. We recommend using more reassuring language to address the customer's concerns." The device displays this feedback to the user, and by allowing the user to review it, the server can improve both customer service skills and emotional understanding skills. When generating feedback, the server can use analytical tools such as Python's Pandas library. The device can also design the feedback screen to be visually appealing using HTML5 and CSS3.

[1087] This system allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. Specifically, by selecting a "product return handling" scenario and role-playing, users can practically learn how to respond to customer emotions. Furthermore, feedback provided after the system is completed provides users with specific guidelines for improving their own responses.

[1088] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1089] Step 1: Server startup and login authentication

[1090] input:

[1091] The server receives the boot command and starts booting the system.

[1092] process:

[1093] The server initializes the basic components (e.g., database connection, user management system, scenario management system) required to provide a learning environment for customer service skills.

[1094] Specific initialization operations include connecting to the MySQL database, starting the Node.js server, and configuring Nginx.

[1095] output:

[1096] The server logs the completion of initialization, generates a login screen, and sends it to the terminal.

[1097] Step 2: Provide login authentication via device

[1098] input:

[1099] The terminal receives the login screen information from the server.

[1100] process:

[1101] The terminal displays a login screen and prompts the user to enter their login ID and password.

[1102] output:

[1103] The user enters their login ID and password, and the device sends the authentication information to the server.

[1104] Step 3: Verifying User Authentication with the Server

[1105] input:

[1106] The server receives the authentication information sent from the terminal.

[1107] process:

[1108] The server checks the user information in the database to verify that the entered authentication information is valid. Specifically, it compares the user ID and hashed password with the information in the database.

[1109] output:

[1110] If authentication is successful, the server generates a scenario selection screen and sends it to the terminal. If authentication fails, it sends an error message to the terminal.

[1111] Step 4: Providing scenario selection on the device

[1112] input:

[1113] The terminal receives the scenario selection screen information from the server.

[1114] process:

[1115] The terminal displays a list of scenarios and prompts the user to select a scenario.

[1116] output:

[1117] The user selects a scenario of interest (e.g., "handling product returns"), and the terminal transmits the selection information to the server.

[1118] Step 5: Prepare scenario data on the server

[1119] input:

[1120] The server receives the scenario selection information transmitted from the terminal.

[1121] process:

[1122] The server loads data corresponding to the selected scenario (e.g., product information, conversation templates, simulation questions) from the database. Specifically, it retrieves the relevant data using an SQL query based on the scenario ID.

[1123] output:

[1124] The server transmits the prepared scenario data to the terminal.

[1125] Step 6: Providing scenario start by terminal

[1126] input:

[1127] The terminal receives the scenario data from the server.

[1128] process:

[1129] The device displays a scenario start screen. Specifically, the scenario start screen is visually generated using a framework such as React.js.

[1130] output:

[1131] Notify the user that the scenario is ready to begin.

[1132] Step 7: Start the role-play and generate prompts with the server

[1133] input:

[1134] The user confirms the start of the scenario.

[1135] process:

[1136] The server runs the scenario and generates key prompts (e.g., "A customer wants to return an item. How would you handle this?") using a generative AI model (e.g., OpenAI GPT-3).

[1137] output:

[1138] The generated prompt is sent to the terminal.

[1139] Step 8: Prompting the Terminal and Getting User Input

[1140] input:

[1141] The terminal receives the prompt received from the server.

[1142] process:

[1143] The terminal displays a prompt, requesting input from the user.

[1144] output:

[1145] The information entered by the user (e.g., "Please tell us the reason for returning the product") is sent from the device to the emotion engine.

[1146] Step 9: Input analysis and emotion data generation by the emotion engine

[1147] input:

[1148] The emotion engine receives the user's input content from the terminal.

[1149] process:

[1150] The emotion engine uses a natural language processing API (e.g., Google Cloud NLP API) to analyze the input and generate emotion data. For example, it can extract emotions such as "anxiety" from the input.

[1151] output:

[1152] The generated emotion data is sent from the emotion engine to the server.

[1153] Step 10: Server Generates and Sends Response

[1154] input:

[1155] The server receives the emotion data sent from the emotion engine.

[1156] process:

[1157] The server generates a response for the customer based on the sentiment analysis results, such as "I'll explain why, but the product was damaged."

[1158] output:

[1159] The generated response is sent to the terminal.

[1160] Step 11: Displaying responses on the device and proceeding with the role-play

[1161] input:

[1162] The terminal receives the response from the server.

[1163] process:

[1164] The terminal displays the response content to the user, and also displays the progress of the simulation to the user.

[1165] output:

[1166] The user enters the next input and the roleplay continues.

[1167] Step 12: Feedback Generation by the Server

[1168] input:

[1169] User behavior history and emotion data when the scenario ends.

[1170] process:

[1171] The server analyzes the user's behavioral history and emotional data and generates feedback, evaluating the data based on various metrics using Python's Pandas library.

[1172] output:

[1173] The generated feedback is sent to the device in JSON format.

[1174] Step 13: Displaying feedback on the device

[1175] input:

[1176] The terminal receives the feedback sent from the server.

[1177] process:

[1178] The device displays the feedback to the user in a visually appealing format using HTML5 and CSS3.

[1179] output:

[1180] The user checks the feedback and uses it for the next simulation.

[1181] This series of processing steps allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. As a specific example, by selecting a "product return handling" scenario and role-playing, users can practically learn how to respond to customers' emotions. In addition, feedback provided after the system is completed provides users with specific guidelines for improving their own responses.

[1182] (Application example 2)

[1183] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1184] Conventional customer service skill learning systems have the problem that it is difficult for users to learn in an environment that closely resembles a real customer service situation. Furthermore, due to the lack of emotion recognition functionality, users are unable to learn how to respond appropriately to customer emotions. Furthermore, feedback is often simple, making it difficult for users to identify specific areas for improvement. The present invention aims to solve these problems and enable users to provide high-quality customer service in practice.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for providing a role-play simulation for the user to learn customer service skills; means for the terminal to perform login authentication of the user; means for the server to verify user authentication; means for the terminal to display scenario options to the user; means for the server to prepare data based on the selected scenario; means for the terminal to display a screen for starting the scenario; means for the server to progress through the scenario and respond to input from the user; emotion engine means for the server to recognize the visitor's emotions in real time; means for the server to adjust the explanation speed and content according to the recognized emotions; means for the server to generate feedback after the scenario ends; and means for the terminal to display the feedback to the user. This enables the user to learn customer service skills that are in line with their emotions in real time and identify specific areas for improvement.

[1186] "Server" refers to a set of computer systems that provide data and services over a computer network.

[1187] "Terminal" refers to a device used by a user to access and operate a computer network.

[1188] "Login authentication" is a procedure for verifying a user's identity when accessing a system.

[1189] "Verifying user authentication" is the process of verifying whether the entered authentication information is correct by checking it against a database.

[1190] "Scenario options" refer to specific learning scenarios that a user can choose from during the learning process.

[1191] "Proceeding through a scenario" means executing a series of learning procedures based on a selected scenario.

[1192] "Emotion engine" refers to software or hardware for analyzing the emotions of users or visitors in real time.

[1193] "Feedback" is an evaluation or advice on improvement provided based on behavior and results after learning or work is completed.

[1194] The system for realizing this invention is composed of a server, a terminal, and an emotion engine. Each component operates as follows.

[1195] 1. Server initialization and login authentication

[1196] The server starts the system and initializes the basic components necessary to provide a role-play simulation environment for learning customer service skills. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters their authentication information, the server checks it against a database to verify that the user is a legitimate user. If authentication is successful, the scenario selection screen is displayed on the terminal.

[1197] 2. Scenario selection and preparation

[1198] The terminal displays a list of scenarios, and the user can select the one they are interested in. For example, scenarios such as "Handling Product Returns" or "Factory Tour Guide" can be selected. The server loads data corresponding to the selected scenario from the database, such as product information, guidance information, conversation templates, and simulation questions. Once the scenario is ready to start, that information is sent to the terminal.

[1199] 3. Role-play initiation and emotion recognition

[1200] The terminal displays a screen to start the scenario, and the server progresses the scenario. For example, in a "Factory Tour Guide" scenario, a prompt such as "We will move to the next tour station. Here, we will explain the manufacturing process" is generated and sent to the terminal. When the user inputs "Tell me more about the manufacturing process," the terminal sends the input to the emotion engine for analysis, which then sends it to the server. Based on the analysis results of the emotion engine, the server generates a response from the customer and sends it to the terminal. For example, if the user inputs something that sounds anxious, a response such as "Don't worry. Our manufacturing process is very safe" is returned.

[1201] 4. Providing Feedback

[1202] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user responded appropriately to the emotions and what areas need improvement. For example, specific advice such as "You conducted the tour appropriately, but next time, be sure to include details about the production line. To alleviate visitors' anxiety, we recommend explaining specific safety measures." The device displays this feedback to the user, who can then confirm it, improving both their customer service skills and their emotional understanding skills.

[1203] To implement this system, the following hardware and software are required:

[1204] Hardware:

[1205] Camera (for facial recognition of visitors)

[1206] Microphone (for voice input)

[1207] Speaker (for audio output)

[1208] A tablet or monitor (to display the interface)

[1209] Emotion recognition system (emotion engine)

[1210] software:

[1211] Python (control program)

[1212] OpenCV (image processing)

[1213] Google Cloud Speech-to-Text (voice recognition)

[1214] TensorFlow (emotion recognition model)

[1215] Flask (web interface)

[1216] For example, if the prompt sentence "Generate the message to be displayed when a visitor shows a confused expression" is input into the generative AI model, the generated message will be "Please feel free to ask us if you have any questions. Let's explain the next station in more detail. Is that okay?" The feedback system can also be improved based on this prompt and message.

[1217] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1218] Step 1:

[1219] The server starts the system and initializes the basic components to provide a role-playing simulation environment for learning customer service skills. The inputs are system configuration information and user data, and the initialized system state is obtained as the output.

[1220] Step 2:

[1221] The terminal presents the user with a login screen and prompts them to enter their login ID and password, which provides the user's authentication information (login ID and password) as input and sends it to the server.

[1222] Step 3:

[1223] The server verifies whether the user is a legitimate user by checking the authentication information it receives against the database. The entered authentication information is compared with the information in the database, and if they match, authentication is successful and the output is that the user is a legitimate user.

[1224] Step 4:

[1225] A scenario selection screen is displayed on the terminal, and the user selects the scenario they are interested in. Here, the input is the user's selection, and the output is the selected scenario information.

[1226] Step 5:

[1227] The server loads relevant data from the database based on the selected scenario and prepares to start the scenario. The input is the selected scenario information, and the output is the scenario data (product information, conversation templates, etc.).

[1228] Step 6:

[1229] The terminal displays the scenario start screen, and the server progresses the scenario. It responds appropriately to input from the user. It provides an environment that is easy for the user to operate by displaying an interface such as prompts. The input is the scenario data and user operations, and the output is the progress of the scenario and the results of the user operations.

[1230] Step 7:

[1231] The server uses an emotion engine to recognize and analyze the user's emotions in real time. The input is the user's voice and video data, and the output is the recognized emotional information. An appropriate response is generated based on this emotional information.

[1232] Step 8:

[1233] The server adjusts the speed and content of the explanation according to the recognized emotion. For example, if the user is confused, the explanation will be slower and more detailed. The input is emotion information and scenario data, and the output is the adjusted explanation content.

[1234] Step 9:

[1235] When the scenario ends, the server generates feedback based on the user's behavioral history and emotional data. The input is the user's operation history and emotional data, and the output is specific feedback content.

[1236] Step 10:

[1237] The device generates feedback and displays it to the user, including specific areas for improvement and success that the user can use to improve their learning experience. The input is the feedback, and the output is the display to the user.

[1238] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1239] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1240] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1241] [Fourth embodiment]

[1242] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1243] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1244] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1245] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1246] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1247] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1248] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1249] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1250] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1251] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1252] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1253] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1254] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1255] The present invention relates to a system that provides a role-play simulation for users to learn customer service skills. This system is composed of a server and a terminal, each of which operates as follows.

[1256] 1. Server startup and login authentication

[1257] The server starts the system and allows the user to access it. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters their authentication information, the server checks it against a database to confirm that they are a legitimate user. If authentication is successful, the user proceeds to the scenario selection screen.

[1258] 2. Scenario selection and preparation

[1259] The terminal displays a list of scenarios, and the user selects the one they are interested in. For example, if they select the "Handling Product Returns" scenario, the server loads the corresponding data (product information, conversation templates, simulation questions, etc.) from the database, and the scenario is ready to begin.

[1260] 3. Begin the role-play

[1261] The terminal displays a screen to start the scenario, and the server runs the scenario. The server generates a prompt such as "A customer wishes to return an item. How would you like to proceed?" and sends it to the terminal. When the user inputs "We would like to know the reason for the return," the server analyzes the input, generates a response from the customer, and sends it to the terminal. For example, a response such as "We will explain the reason. The item was damaged" is returned.

[1262] 4. Providing Feedback

[1263] When the scenario ends, the server analyzes the user's overall response and generates feedback. The feedback includes what went well and what needs improvement. For example, the server might say, "We properly confirmed the reason for the return, but please remember to suggest a coupon next time." The terminal displays this feedback for the user to review.

[1264] This system allows users to efficiently improve their customer service skills by simulating actual customer service situations. In particular, the feedback function allows users to identify specific areas for improvement and reflect them in their actual work.

[1265] The processing flow will be explained below.

[1266] Step 1:

[1267] The server starts the system. The server initializes the basic components of the role-play simulation system and prepares to generate the login screen.

[1268] Step 2:

[1269] The terminal displays a login screen to the user. The terminal provides the user with a form to enter their login ID and password, and includes a "Login" button.

[1270] Step 3:

[1271] The user enters their login credentials. The user enters their login ID and password and clicks the "Login" button.

[1272] Step 4:

[1273] The server receives the user's authentication information and verifies the login ID and password against the database.

[1274] Step 5:

[1275] The server sends the authentication result to the terminal. If the authentication is successful, the server sends an instruction to the terminal to display the scenario selection screen.

[1276] Step 6:

[1277] The terminal displays a scenario selection screen. The terminal presents the user with a list of scenarios, offering options such as "handling product returns."

[1278] Step 7:

[1279] The user selects a scenario: the user clicks on the scenario that interests them.

[1280] Step 8:

[1281] The server loads data based on the selected scenario, and retrieves relevant product information, conversation templates, simulation questions, etc. from the database.

[1282] Step 9:

[1283] The server notifies the terminal that it is ready to start the scenario. The server sends a command to the terminal to start the scenario.

[1284] Step 10:

[1285] The terminal displays the scenario start screen. The terminal displays a screen that explains the scenario's outline and purpose to the user.

[1286] Step 11:

[1287] The server runs the scenario. It generates a prompt such as "The customer wants to return the product. How would you like to proceed?" and sends it to the terminal.

[1288] Step 12:

[1289] The user enters a response, such as "We would like to ask you the reason for the return," in the chat box and presses the send button.

[1290] Step 13:

[1291] The server processes the user's input. The server analyzes the user's input and generates a response depending on the progress of the scenario.

[1292] Step 14:

[1293] The server sends a response to the terminal, for example, "I'll explain why. The product was damaged."

[1294] Step 15:

[1295] The terminal displays the response to the user. The terminal displays the response received from the server in a chat box.

[1296] Step 16:

[1297] When the scenario ends, the server generates feedback by analyzing the user's response history and creating feedback that includes what went well and what needs to be improved.

[1298] Step 17:

[1299] The server sends the feedback to the device. The server sends the generated feedback to the device.

[1300] Step 18:

[1301] The terminal displays the feedback. The terminal visualizes the feedback sent from the server to the user.

[1302] Example 1

[1303] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1304] Conventional customer service skill learning systems have difficulty simulating actual customer service situations, making it difficult to effectively improve the skills required for real-world work. In particular, it is difficult to generate appropriate responses from a customer role in real time in response to user input and provide feedback based on that, which limits the effectiveness of learning.

[1305] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1306] In this invention, the server includes means for providing a role-playing simulation for a user to learn customer service skills, means for the server to analyze user input using a natural language processing library, means for the server to generate responses for the customer role using a generative AI model, and means for the server to generate feedback using an analytical model after the scenario ends. This allows users to effectively improve their customer service skills through a simulation that resembles a real-life customer service situation, and further allows them to receive specific feedback on their responses.

[1307] A "user" is a person who uses the system to learn customer service techniques.

[1308] "Customer service skills" refers to the skills and knowledge required to deal with customers.

[1309] "Role-play simulation" is a system that allows users to practice by simulating actual customer service situations.

[1310] A "server" is a computer device that performs the main processing of a system.

[1311] A "terminal" is a computer device or device that allows a user to operate the system.

[1312] "Login authentication" is the procedure for verifying a user's authentication information and confirming their access rights to the system.

[1313] "Authentication information" refers to information used when logging in, such as a user ID and password.

[1314] A "scenario" is a pre-set story that simulates a specific customer service situation.

[1315] "Scenario Choices" are options that allow the user to select the scenario they want to practice.

[1316] A "natural language processing library" is a program library for analyzing human language.

[1317] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on user input.

[1318] A "prompt" is text that prompts the user for the next action or response.

[1319] A "database" is a system for organizing, storing, and retrieving data.

[1320] "Feedback" refers to an evaluation of the user's response and areas for improvement.

[1321] An "analytical model" is an algorithm or program used to evaluate a user's performance.

[1322] The present invention relates to a system that provides role-play simulations for users to learn customer service skills. This system is comprised of a server and a terminal, and operates as follows.

[1323] Server startup and login authentication

[1324] The server starts the system and establishes a connection with the database. For example, a backend server built on a cloud service (e.g., a Node.js server on AWS) starts and connects to the MySQL database. The terminal displays a login screen to the user, and the user enters a login ID and password. For example, the user enters "user123" and "pass123." The server receives the entered authentication information and executes an SQL query to compare it with the database. If authentication is successful, the server returns the data of the scenario selection screen to the terminal, which displays it.

[1325] Scenario selection and preparation

[1326] The terminal displays a list of scenarios to the user. When the user selects the "Handling Product Returns" scenario, the selection information is sent to the server. The server loads data corresponding to the selected scenario (e.g., detailed product information, conversation templates, simulation questions, etc.) from the database. For example, the query "SELECT FROM scenarios WHERE scenario_name="Handling Product Returns";" is used for this purpose. The server caches the loaded data, and when it is ready, displays the scenario start screen on the terminal.

[1327] Beginning the role-play

[1328] The device displays a scenario start screen to the user and presents the initial prompt. In response, for example, the text "A customer wishes to return an item. How would you like to proceed?" is displayed. The user enters "I would like to know the reason for the return" and presses the send button. The server receives the input and analyzes it using a natural language processing library (e.g., spaCy or NLTK). A generative AI model (e.g., GPT-3) is used to generate a response for the customer, such as "I will explain the reason. The item was damaged." The generated response is then sent to the device, which displays it.

[1329] Providing Feedback

[1330] After the scenario is over, the server analyzes the user's response. For example, it analyzes the user's response log and uses a generative AI model to perform an overall evaluation. Specific feedback is generated, such as a message like, "We've properly confirmed the reason for the return, but please remember to suggest a coupon next time." The terminal displays the generated feedback to the user, who can then review it.

[1331] In this way, the system of the present invention allows users to simulate customer service situations in an environment that closely resembles a real customer service situation, effectively improving their customer service skills. In addition, by receiving specific feedback, users can improve their own customer service methods and reflect them in their actual work.

[1332] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1333] Step 1:

[1334] The server starts the system and establishes a connection to the database. The server starts a backend Node.js server and connects to the MySQL database. The system is now up and running and ready to accept user access.

[1335] Step 2:

[1336] The terminal displays a login screen and asks the user to enter authentication information. The user enters their login ID and password. Input: User's login ID and password. Output: User's authentication information is sent to the server.

[1337] Step 3:

[1338] The server checks the received authentication information against the database. The server executes an SQL query (e.g. SELECT FROM users WHERE user_id='user123' AND password='pass123';) to verify the authentication information. Input: User authentication information. Output: Authentication result (success or failure) is sent to the terminal.

[1339] Step 4:

[1340] The terminal displays the following screen to the user based on the authentication result. If authentication is successful, the terminal displays the scenario selection screen. Input: Authentication result. Output: Scenario selection screen.

[1341] Step 5:

[1342] The terminal displays a list of scenarios and lets the user select one. For example, the user selects the "Handling product returns" scenario. Input: User's scenario selection. Output: Selected scenario information is sent to the server.

[1343] Step 6:

[1344] The server loads data corresponding to the selected scenario from the database. For example, the server executes a query such as SELECT FROM scenarios WHERE scenario_name="Product return handling"; Input: Selected scenario information. Output: Corresponding scenario data is sent to the terminal.

[1345] Step 7:

[1346] The terminal displays the scenario start screen and presents the first prompt to the user. For example, the prompt may read, "A customer wishes to return an item. How would you like to proceed?" Input: Scenario data. Output: The first prompt is displayed to the user.

[1347] Step 8:

[1348] The user enters a response in the input box and submits it. For example, "Please tell us the reason for the return." Input: User's response. Output: User's response is sent to the server.

[1349] Step 9:

[1350] The server parses the user response using a natural language processing library, such as spaCy or NLTK. Input: User response. Output: Parsed data.

[1351] Step 10:

[1352] The server uses a generative AI model to generate a response for the customer. For example, it uses GPT-3 to generate a response like "I'll explain why. The item was damaged." Input: Parsed data. Output: Generated response for the customer.

[1353] Step 11:

[1354] The terminal displays the generated customer response to the user. Input: The generated customer response. Output: The customer response is displayed to the user.

[1355] Step 12:

[1356] After the scenario is completed, the server analyzes the user's responses and generates feedback. The analytical model is used to score the user's response log and identify areas for improvement. Input: User's response log. Output: Generated feedback.

[1357] Step 13:

[1358] The terminal displays the generated feedback to the user. For example, "We've properly verified the reason for the return, but don't forget to offer a coupon next time." Input: The generated feedback. Output: The feedback is displayed to the user.

[1359] This process allows users to simulate actual customer service situations and effectively learn customer service techniques. Through more specific responses and feedback, users can acquire skills that can be applied to actual work.

[1360] (Application example 1)

[1361] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1362] Conventional role-play simulation systems have limited methods for providing appropriate feedback to users learning customer service skills. As a result, users experience insufficient learning through the simulation, making it difficult to identify specific areas for improvement that can be applied in practice. Furthermore, while conventional systems have a general feedback function, they lack dynamic and personalized feedback based on input responses. This makes it difficult for users to learn the optimal response methods for each specific scenario. Furthermore, basic functions such as login authentication and scenario selection have also presented issues in terms of security and convenience.

[1363] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1364] In this invention, the server includes an information processing device that provides a role-playing simulation for a user to learn customer service skills; an operation terminal that performs user login authentication; an information processing device that verifies user authentication; an operation terminal that displays scenario options to the user; an information processing device that organizes data based on the selected scenario; an operation terminal that displays a screen indicating the start of the scenario; an information processing device that progresses through the scenario and responds to input from the user; an information processing device that generates feedback after the scenario ends; an operation terminal that displays the feedback to the user; a means for using a generative AI model to generate feedback based on user responses input from the operation terminal; and a means for inputting prompt sentences into the generative AI model. This allows the user to receive dynamic and personalized feedback in an interactive format and identify specific areas for improvement. This also provides high security and convenience, enabling effective and efficient improvement of customer service skills.

[1365] "User" refers to a person who uses the system to learn customer service skills, or the user of the system.

[1366] "Customer service skills" refer to the techniques and abilities required for dealing with customers, and are particularly required in customer service work in physical stores.

[1367] "Role-play simulation" refers to a simulation that simulates actual customer service work based on a specific scenario.

[1368] "Information processing device" refers to a computer device such as a server for processing and analyzing data, progressing a scenario, and the like.

[1369] An "operation terminal" is a device that is directly operated by a user, and includes mobile devices such as smartphones and tablets.

[1370] "Login authentication" refers to the process by which a user enters authentication information such as an ID and password to verify their identity in order to gain permission to access a system.

[1371] "User authentication" is the procedure in which the server verifies the authentication information entered by the user and confirms that the user is a legitimate user.

[1372] "Scenario options" refers to a list of multiple customer service scenarios that the user can experience, each of which simulates a specific situation.

[1373] "Preparing data" means preparing the information and settings required for the selected scenario and bringing it into a state ready for starting the simulation.

[1374] The "scenario start screen" refers to a screen that notifies the user of the start of a simulation.

[1375] A "generative AI model" is an artificial intelligence model that generates feedback and responses based on user input, and uses techniques such as machine learning.

[1376] A "prompt" is a question or instruction that tells the user what to do next.

[1377] The present invention relates to a system that provides role-play simulations for users to learn customer service skills. This system is composed of an information processing device (server) and an operation terminal (smartphone, tablet, etc.), and each operates as follows.

[1378] 1. User authentication:

[1379] The user accesses the system from an operation terminal and performs login authentication. When the user enters their ID and password, the information processing device verifies this and confirms that they are a legitimate user. If authentication is successful, the scenario selection screen is displayed.

[1380] 2. Scenario Selection:

[1381] The operation terminal displays multiple customer service scenarios (e.g., "handling product returns," "handling complaints," "product suggestions," etc.) to the user. When the user selects a scenario that interests them, the information processing device prepares the corresponding data (product information, conversation templates, simulation questions, etc.).

[1382] 3. Begin the role-play simulation:

[1383] The operating terminal displays a screen for starting the scenario, and interactive questions based on the scenario are displayed. The information processing device analyzes the user's input according to the progress of the scenario and generates an appropriate reply. For example, a prompt such as "A customer has complained about a defective product. How would you respond first?" is displayed. When the user enters a response, the information processing device analyzes it and generates the next response.

[1384] 4. Providing Feedback:

[1385] At the end of the scenario, the information processing device analyzes the user's overall response and generates feedback using a generative AI model. The feedback includes what was appropriate and what needs improvement. For example, specific advice such as "We properly confirmed the reason for the return, but don't forget to suggest a coupon next time" is provided. The operating device displays this feedback, allowing the user to confirm it.

[1386] Hardware and software used:

[1387] Hardware: smartphones, tablets, servers

[1388] Software: Python, API server, database, generative AI model

[1389] Examples:

[1390] The user opens the application and logs in by entering their ID and password. After logging in, they select the "Complaint Handling" scenario. The scenario begins and a prompt appears: "A customer has complained about a defective product. What is your first response?" The user enters their response and practices handling a complaint through a simulation. After the scenario ends, they review the provided feedback and improve their future responses.

[1391] In this way, users can efficiently improve their customer service skills by simulating actual customer service situations. Furthermore, interactive feedback allows users to identify specific areas for improvement and reflect them in their actual work.

[1392] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1393] Step 1:

[1394] (Login authentication)

[1395] The user enters an ID and password from the operating terminal. This is the input data. The terminal sends this authentication information to the server. The server verifies the authentication information against a database and confirms that the user is legitimate. If authentication is successful, the server generates a session token and sends it to the terminal. The terminal displays a scenario selection screen to the user. The output is the session token and the scenario selection screen.

[1396] Step 2:

[1397] (Scenario selection)

[1398] The user selects a scenario from the operating terminal. This is the input data. The terminal sends the selected scenario ID to the server. The server then prepares and retrieves the corresponding data (product information, conversation templates, simulation questions, etc.) from the database. The output is the prepared data.

[1399] Step 3:

[1400] (Scenario start screen display)

[1401] The terminal displays a scenario start screen to the user. A prompt sentence based on the scenario is displayed first. The input is the prepared data, and the output is the scenario start screen and the prompt sentence.

[1402] Step 4:

[1403] (Role-play simulation progress)

[1404] The user responds to the prompt sentence from the operating terminal. This response is the input data. The terminal sends this response to the server. The server uses a generative AI model to analyze the user's response and generate an appropriate reply. The generated reply is sent to the terminal and displayed to the user again. The output is the generated reply.

[1405] Step 5:

[1406] (Scenario end decision)

[1407] The server monitors the progress of the scenario and determines whether the scenario has finished. The progress of the scenario is the input data, and the decision on whether the scenario has finished is the output.

[1408] Step 6:

[1409] (Generating feedback)

[1410] Once the scenario is over, the server analyzes the user's overall response and generates feedback using a generative AI model, including what went well and what needs improvement. The input is the user's response log, and the output is the generated feedback.

[1411] Step 7:

[1412] (Show feedback)

[1413] The terminal displays the generated feedback to the user. The input is the generated feedback, and the output is the feedback display screen. The user can check the specific advice and improve their future actions.

[1414] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1415] The present invention relates to a system that provides a role-play simulation for users to learn customer service skills, and also combines it with an emotion engine that recognizes the user's emotions. This system is composed of a server means, a terminal means, and an emotion engine, each of which operates as follows.

[1416] 1. Server startup and login authentication

[1417] The server starts the system and initializes the basic components necessary to provide a learning environment for customer service skills. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters this authentication information, the server checks it against a database to confirm whether the user is a legitimate user. If authentication is successful, the scenario selection screen is displayed on the terminal.

[1418] 2. Scenario selection and preparation

[1419] The terminal displays a list of scenarios, and the user selects one that interests them, such as "Handling Product Returns." The server loads data corresponding to the selected scenario, such as product information, conversation templates, and simulation questions, from the database. Once the scenario is ready to start, that information is sent to the terminal.

[1420] 3. Role-play initiation and emotion recognition

[1421] The terminal displays a screen to start the scenario, and the server progresses the scenario. For example, a prompt such as "A customer wishes to return an item. How would you respond?" is generated and sent to the terminal. When the user inputs "I would like to know the reason for the return," the terminal sends the input to the emotion engine for analysis, which then sends the result to the server. Based on the analysis results of the emotion engine, the server generates a response for the customer and sends it to the terminal. For example, if the user inputs with an anxious look on their face, a response such as "I will explain the reason, but the item was damaged" is returned.

[1422] 4. Providing Feedback

[1423] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user handled the emotion appropriately and what areas need improvement. For example, specific advice such as "We properly confirmed the reason for the return, but next time, don't forget to offer a coupon. We recommend using more reassuring language to address the customer's concerns" may be provided. The device displays this feedback to the user, and by the user confirming it, the device can improve both customer service skills and emotional understanding skills.

[1424] This system allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. In particular, the addition of emotion recognition functionality allows users to learn how to respond to customers in accordance with their emotions, enabling them to provide high-quality service in real-world situations.

[1425] The processing flow will be explained below.

[1426] Step 1:

[1427] The server starts the system. The server initializes the basic components of the role-play simulation system and prepares to generate the login screen.

[1428] Step 2:

[1429] The terminal displays a login screen to the user. The terminal provides the user with a form to enter their login ID and password, and includes a "Login" button.

[1430] Step 3:

[1431] The user enters their login credentials. The user enters their login ID and password and clicks the "Login" button.

[1432] Step 4:

[1433] The server receives the user's authentication information and verifies the login ID and password against the database.

[1434] Step 5:

[1435] The server sends the authentication result to the terminal. If the authentication is successful, the server sends an instruction to the terminal to display the scenario selection screen.

[1436] Step 6:

[1437] The terminal displays a scenario selection screen. The terminal presents the user with a list of scenarios, offering options such as "handling product returns."

[1438] Step 7:

[1439] The user selects a scenario: the user clicks on the scenario that interests them.

[1440] Step 8:

[1441] The server loads data based on the selected scenario, and retrieves relevant product information, conversation templates, simulation questions, etc. from the database.

[1442] Step 9:

[1443] The server notifies the terminal that it is ready to start the scenario. The server sends a command to the terminal to start the scenario.

[1444] Step 10:

[1445] The terminal displays the scenario start screen. The terminal displays a screen that explains the scenario's outline and purpose to the user.

[1446] Step 11:

[1447] The server runs the scenario. It generates a prompt such as "The customer wants to return the product. How would you like to proceed?" and sends it to the terminal.

[1448] Step 12:

[1449] The user enters a response, such as "We would like to ask you the reason for the return," in the chat box and presses the send button.

[1450] Step 13:

[1451] The device sends the user's input to the emotion engine, which analyzes the input and tone of the words to identify the user's emotion.

[1452] Step 14:

[1453] The emotion engine sends the analysis results to the server, which include emotional information such as whether the user is anxious, angry, or relieved.

[1454] Step 15:

[1455] The server generates a response based on the user's emotional state: for example, if the user appears anxious, it generates a reassuring response such as, "The customer will explain why, but the item was damaged."

[1456] Step 16:

[1457] The server sends a response to the terminal. The generated response is sent to the terminal.

[1458] Step 17:

[1459] The terminal displays the response to the user. The terminal displays the response received from the server in a chat box.

[1460] Step 18:

[1461] When the scenario ends, the server generates feedback, summarizing the evaluation and areas for improvement based on the user's response history and the analysis results of the emotion engine.

[1462] Step 19:

[1463] The server sends the feedback to the terminal. The generated feedback is sent to the terminal.

[1464] Step 20:

[1465] The terminal displays the feedback. The terminal visualizes the feedback sent from the server to the user.

[1466] Example 2

[1467] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1468] In conventional customer service skill learning systems, it was difficult for users to learn appropriate responses based on actual customer emotional states. This resulted in problems such as insufficient acquisition of practical skills to improve on-site customer service skills and customer satisfaction. Furthermore, there was a lack of mechanisms to provide specific feedback on user behavior.

[1469] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1470] In this invention, the server includes computer means for providing a role-play simulation for a user to learn customer service skills, means for the terminal to perform login authentication of the user, means for the computer to verify user authentication, means for the terminal to display scenario options to the user, means for the computer to prepare data based on the selected scenario, means for the terminal to display a screen for starting the scenario, means for the computer to progress through the scenario and respond to input from the user, means for the terminal to send user input to an emotion engine, means for the emotion engine to analyze the user input and generate emotion data, means for the computer to generate a response based on the emotion analysis result, means for the computer to generate feedback after the scenario ends, and means for the terminal to display the feedback to the user. This enables the user to learn customer service skills tailored to the emotional states of actual customers and receive specific feedback.

[1471] "User" refers to a learner who uses the system to learn customer service skills.

[1472] "Computer means" refers to servers and cloud services that provide role-play simulations of customer service skills.

[1473] "Terminal" refers to a device such as a PC, tablet, or smartphone that a user uses to access the system.

[1474] "Login authentication" is the procedure for authenticating access to a system using a user's ID and password.

[1475] "User authentication verification" refers to the process of verifying whether the entered authentication information is legitimate.

[1476] "Scenario options" refers to multiple role-play scenarios that a user can choose from to learn.

[1477] "Data preparation" refers to a series of processes that load various information from a database based on a selected scenario.

[1478] The "scenario start screen" refers to the screen that is displayed to the user when the scenario starts.

[1479] "Scenario progression" refers to the process of managing the execution of the role-play simulation and the system's response to user input.

[1480] "Emotion engine" refers to software or algorithms that analyze user input and assess emotional state.

[1481] "Emotion data" refers to emotional information obtained by the emotion engine through analysis of user input.

[1482] "Response generation" refers to the process of creating an appropriate response to the user based on the analysis results of the emotion engine.

[1483] "Feedback" refers to information that provides an evaluation of the user's actions and decisions and specific advice after the scenario is completed.

[1484] "Displaying feedback" refers to presenting the generated feedback content to the user.

[1485] This invention relates to a system that provides a role-play simulation for users to learn customer service skills, and also combines it with an emotion engine that recognizes the user's emotions. The purpose of this system is to enable users to efficiently improve their customer service skills and emotional understanding in an environment that is similar to a real customer service situation. Specifically, the system is composed of a server, a terminal, and an emotion engine.

[1486] Server startup and login authentication

[1487] The server starts the system and initializes the basic components necessary to provide a learning environment for customer service skills. At this time, the server uses multiple components, including database connections, a user management system, and a scenario management system. Specifically, it uses a MySQL database, a Node.js server, and Nginx. The terminal provides the user with a login screen and prompts them to enter their login ID and password. When the user enters this authentication information, the server checks it against the database to confirm whether the user is a legitimate user. For example, it compares the hash values ​​of the user ID and password stored in the database. If authentication is successful, the scenario selection screen is displayed on the terminal.

[1488] Scenario selection and preparation

[1489] The device displays a list of scenarios, and the user selects the one they are interested in. For example, if the user selects a scenario such as "Handling Product Returns," the device sends the user's selection to the server. The server loads data corresponding to the selected scenario, such as product information, conversation templates, and simulation questions, from a database. At this time, the server efficiently retrieves the data using SQL queries, etc. Once the scenario is ready to start, that information is sent to the device. A framework such as React.js can be used by the device to display the scenario start screen.

[1490] Role-play initiation and emotion recognition

[1491] The device displays a screen to start the scenario, and the server runs the scenario. For example, a prompt such as "A customer wishes to return a product. How would you respond?" is generated and sent to the device. When generating this prompt, the server can use a generative AI model (e.g., OpenAI GPT-3). When the user enters "I would like to know the reason for the return," the device sends the input to an emotion engine for analysis. The emotion engine analyzes the input using Google Cloud's natural language processing API or similar and generates emotion data. For example, this emotion data may determine that the input expresses "anxiety." The analysis results are sent to the server, which generates a response based on the emotion analysis results and sends it to the device. For example, a response such as "I will explain the reason, but the product was damaged" may be returned.

[1492] Providing Feedback

[1493] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user appropriately responded to the user's emotions and what areas need improvement. For example, the server might provide specific advice such as, "We properly confirmed the reason for the return, but next time, please remember to offer a coupon. We recommend using more reassuring language to address the customer's concerns." The device displays this feedback to the user, and by allowing the user to review it, the server can improve both customer service skills and emotional understanding skills. When generating feedback, the server can use analytical tools such as Python's Pandas library. The device can also design the feedback screen to be visually appealing using HTML5 and CSS3.

[1494] This system allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. Specifically, by selecting a "product return handling" scenario and role-playing, users can practically learn how to respond to customer emotions. Furthermore, feedback provided after the system is completed provides users with specific guidelines for improving their own responses.

[1495] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1496] Step 1: Server startup and login authentication

[1497] input:

[1498] The server receives the boot command and starts booting the system.

[1499] process:

[1500] The server initializes the basic components (e.g., database connection, user management system, scenario management system) required to provide a learning environment for customer service skills.

[1501] Specific initialization operations include connecting to the MySQL database, starting the Node.js server, and configuring Nginx.

[1502] output:

[1503] The server logs the completion of initialization, generates a login screen, and sends it to the terminal.

[1504] Step 2: Provide login authentication via device

[1505] input:

[1506] The terminal receives the login screen information from the server.

[1507] process:

[1508] The terminal displays a login screen and prompts the user to enter their login ID and password.

[1509] output:

[1510] The user enters their login ID and password, and the device sends the authentication information to the server.

[1511] Step 3: Verifying User Authentication with the Server

[1512] input:

[1513] The server receives the authentication information sent from the terminal.

[1514] process:

[1515] The server checks the user information in the database to verify that the entered authentication information is valid. Specifically, it compares the user ID and hashed password with the information in the database.

[1516] output:

[1517] If authentication is successful, the server generates a scenario selection screen and sends it to the terminal. If authentication fails, it sends an error message to the terminal.

[1518] Step 4: Providing scenario selection on the device

[1519] input:

[1520] The terminal receives the scenario selection screen information from the server.

[1521] process:

[1522] The terminal displays a list of scenarios and prompts the user to select a scenario.

[1523] output:

[1524] The user selects a scenario of interest (e.g., "handling product returns"), and the terminal transmits the selection information to the server.

[1525] Step 5: Prepare scenario data on the server

[1526] input:

[1527] The server receives the scenario selection information transmitted from the terminal.

[1528] process:

[1529] The server loads data corresponding to the selected scenario (e.g., product information, conversation templates, simulation questions) from the database. Specifically, it retrieves the relevant data using an SQL query based on the scenario ID.

[1530] output:

[1531] The server transmits the prepared scenario data to the terminal.

[1532] Step 6: Providing scenario start by terminal

[1533] input:

[1534] The terminal receives the scenario data from the server.

[1535] process:

[1536] The device displays a scenario start screen. Specifically, the scenario start screen is visually generated using a framework such as React.js.

[1537] output:

[1538] Notify the user that the scenario is ready to begin.

[1539] Step 7: Start the role-play and generate prompts with the server

[1540] input:

[1541] The user confirms the start of the scenario.

[1542] process:

[1543] The server runs the scenario and generates key prompts (e.g., "A customer wants to return an item. How would you handle this?") using a generative AI model (e.g., OpenAI GPT-3).

[1544] output:

[1545] The generated prompt is sent to the terminal.

[1546] Step 8: Prompting the Terminal and Getting User Input

[1547] input:

[1548] The terminal receives the prompt received from the server.

[1549] process:

[1550] The terminal displays a prompt, requesting input from the user.

[1551] output:

[1552] The information entered by the user (e.g., "Please tell us the reason for returning the product") is sent from the device to the emotion engine.

[1553] Step 9: Input analysis and emotion data generation by the emotion engine

[1554] input:

[1555] The emotion engine receives the user's input content from the terminal.

[1556] process:

[1557] The emotion engine uses a natural language processing API (e.g., Google Cloud NLP API) to analyze the input and generate emotion data. For example, it can extract emotions such as "anxiety" from the input.

[1558] output:

[1559] The generated emotion data is sent from the emotion engine to the server.

[1560] Step 10: Server Generates and Sends Response

[1561] input:

[1562] The server receives the emotion data sent from the emotion engine.

[1563] process:

[1564] The server generates a response for the customer based on the sentiment analysis results, such as "I'll explain why, but the product was damaged."

[1565] output:

[1566] The generated response is sent to the terminal.

[1567] Step 11: Displaying responses on the device and proceeding with the role-play

[1568] input:

[1569] The terminal receives the response from the server.

[1570] process:

[1571] The terminal displays the response content to the user, and also displays the progress of the simulation to the user.

[1572] output:

[1573] The user enters the next input and the roleplay continues.

[1574] Step 12: Feedback Generation by the Server

[1575] input:

[1576] User behavior history and emotion data when the scenario ends.

[1577] process:

[1578] The server analyzes the user's behavioral history and emotional data and generates feedback, evaluating the data based on various metrics using Python's Pandas library.

[1579] output:

[1580] The generated feedback is sent to the device in JSON format.

[1581] Step 13: Displaying feedback on the device

[1582] input:

[1583] The terminal receives the feedback sent from the server.

[1584] process:

[1585] The device displays the feedback to the user in a visually appealing format using HTML5 and CSS3.

[1586] output:

[1587] The user checks the feedback and uses it for the next simulation.

[1588] This series of processing steps allows users to efficiently improve their customer service skills and emotional understanding in an environment that closely resembles a real customer service situation. As a specific example, by selecting a "product return handling" scenario and role-playing, users can practically learn how to respond to customers' emotions. In addition, feedback provided after the system is completed provides users with specific guidelines for improving their own responses.

[1589] (Application example 2)

[1590] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1591] Conventional customer service skill learning systems have the problem that it is difficult for users to learn in an environment that closely resembles a real customer service situation. Furthermore, due to the lack of emotion recognition functionality, users are unable to learn how to respond appropriately to customer emotions. Furthermore, feedback is often simple, making it difficult for users to identify specific areas for improvement. The present invention aims to solve these problems and enable users to provide high-quality customer service in practice.

[1592] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for providing a role-play simulation for the user to learn customer service skills; means for the terminal to perform login authentication of the user; means for the server to verify user authentication; means for the terminal to display scenario options to the user; means for the server to prepare data based on the selected scenario; means for the terminal to display a screen for starting the scenario; means for the server to progress through the scenario and respond to input from the user; emotion engine means for the server to recognize the visitor's emotions in real time; means for the server to adjust the explanation speed and content according to the recognized emotions; means for the server to generate feedback after the scenario ends; and means for the terminal to display the feedback to the user. This enables the user to learn customer service skills that are in line with their emotions in real time and identify specific areas for improvement.

[1593] "Server" refers to a set of computer systems that provide data and services over a computer network.

[1594] "Terminal" refers to a device used by a user to access and operate a computer network.

[1595] "Login authentication" is a procedure for verifying a user's identity when accessing a system.

[1596] "Verifying user authentication" is the process of verifying whether the entered authentication information is correct by checking it against a database.

[1597] "Scenario options" refer to specific learning scenarios that a user can choose from during the learning process.

[1598] "Proceeding through a scenario" means executing a series of learning procedures based on a selected scenario.

[1599] "Emotion engine" refers to software or hardware for analyzing the emotions of users or visitors in real time.

[1600] "Feedback" is an evaluation or advice on improvement provided based on behavior and results after learning or work is completed.

[1601] The system for realizing this invention is composed of a server, a terminal, and an emotion engine. Each component operates as follows.

[1602] 1. Server initialization and login authentication

[1603] The server starts the system and initializes the basic components necessary to provide a role-play simulation environment for learning customer service skills. The terminal presents the user with a login screen and prompts them to enter their login ID and password. When the user enters their authentication information, the server checks it against a database to verify that the user is a legitimate user. If authentication is successful, the scenario selection screen is displayed on the terminal.

[1604] 2. Scenario selection and preparation

[1605] The terminal displays a list of scenarios, and the user can select the one they are interested in. For example, scenarios such as "Handling Product Returns" or "Factory Tour Guide" can be selected. The server loads data corresponding to the selected scenario from the database, such as product information, guidance information, conversation templates, and simulation questions. Once the scenario is ready to start, that information is sent to the terminal.

[1606] 3. Role-play initiation and emotion recognition

[1607] The terminal displays a screen to start the scenario, and the server progresses the scenario. For example, in a "Factory Tour Guide" scenario, a prompt such as "We will move to the next tour station. Here, we will explain the manufacturing process" is generated and sent to the terminal. When the user inputs "Tell me more about the manufacturing process," the terminal sends the input to the emotion engine for analysis, which then sends it to the server. Based on the analysis results of the emotion engine, the server generates a response from the customer and sends it to the terminal. For example, if the user inputs something that sounds anxious, a response such as "Don't worry. Our manufacturing process is very safe" is returned.

[1608] 4. Providing Feedback

[1609] Once the scenario is over, the server generates feedback based on the user's behavioral history and emotional data. The feedback includes how the user responded appropriately to the emotions and what areas need improvement. For example, specific advice such as "You conducted the tour appropriately, but next time, be sure to include details about the production line. To alleviate visitors' anxiety, we recommend explaining specific safety measures." The device displays this feedback to the user, who can then confirm it, improving both their customer service skills and their emotional understanding skills.

[1610] To implement this system, the following hardware and software are required:

[1611] Hardware:

[1612] Camera (for facial recognition of visitors)

[1613] Microphone (for voice input)

[1614] Speaker (for audio output)

[1615] A tablet or monitor (to display the interface)

[1616] Emotion recognition system (emotion engine)

[1617] software:

[1618] Python (control program)

[1619] OpenCV (image processing)

[1620] Google Cloud Speech-to-Text (voice recognition)

[1621] TensorFlow (emotion recognition model)

[1622] Flask (web interface)

[1623] For example, if the prompt sentence "Generate the message to be displayed when a visitor shows a confused expression" is input into the generative AI model, the generated message will be "Please feel free to ask us if you have any questions. Let's explain the next station in more detail. Is that okay?" The feedback system can also be improved based on this prompt and message.

[1624] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1625] Step 1:

[1626] The server starts the system and initializes the basic components to provide a role-playing simulation environment for learning customer service skills. The inputs are system configuration information and user data, and the initialized system state is obtained as the output.

[1627] Step 2:

[1628] The terminal presents the user with a login screen and prompts them to enter their login ID and password, which provides the user's authentication information (login ID and password) as input and sends it to the server.

[1629] Step 3:

[1630] The server verifies whether the user is a legitimate user by checking the authentication information it receives against the database. The entered authentication information is compared with the information in the database, and if they match, authentication is successful and the output is that the user is a legitimate user.

[1631] Step 4:

[1632] A scenario selection screen is displayed on the terminal, and the user selects the scenario they are interested in. Here, the input is the user's selection, and the output is the selected scenario information.

[1633] Step 5:

[1634] The server loads relevant data from the database based on the selected scenario and prepares to start the scenario. The input is the selected scenario information, and the output is the scenario data (product information, conversation templates, etc.).

[1635] Step 6:

[1636] The terminal displays the scenario start screen, and the server progresses the scenario. It responds appropriately to input from the user. It provides an environment that is easy for the user to operate by displaying an interface such as prompts. The input is the scenario data and user operations, and the output is the progress of the scenario and the results of the user operations.

[1637] Step 7:

[1638] The server uses an emotion engine to recognize and analyze the user's emotions in real time. The input is the user's voice and video data, and the output is the recognized emotional information. An appropriate response is generated based on this emotional information.

[1639] Step 8:

[1640] The server adjusts the speed and content of the explanation according to the recognized emotion. For example, if the user is confused, the explanation will be slower and more detailed. The input is emotion information and scenario data, and the output is the adjusted explanation content.

[1641] Step 9:

[1642] When the scenario ends, the server generates feedback based on the user's behavioral history and emotional data. The input is the user's operation history and emotional data, and the output is specific feedback content.

[1643] Step 10:

[1644] The device generates feedback and displays it to the user, including specific areas for improvement and success that the user can use to improve their learning experience. The input is the feedback, and the output is the display to the user.

[1645] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1646] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1647] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1648] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1649] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1650] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1651] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1652] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1653] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1654] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1655] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1656] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1657] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1658] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1659] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1660] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1661] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1662] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1663] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1664] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1665] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1666] The following is further disclosed regarding the above embodiment.

[1667] (Claim 1)

[1668] a server means for providing a role-play simulation for users to learn customer service skills;

[1669] A means for the terminal to perform login authentication of a user;

[1670] a means by which the server verifies user authentication;

[1671] a means for the terminal to display scenario options to the user;

[1672] means for the server to prepare data based on the selected scenario;

[1673] A means for the terminal to display a screen for starting a scenario;

[1674] A means for the server to progress the scenario and respond to input from the user;

[1675] A means for the server to generate feedback after the scenario is completed;

[1676] means for the terminal to display feedback to the user;

[1677] A system including:

[1678] (Claim 2)

[1679] 2. The system of claim 1, wherein the server further comprises means for analyzing a user's input and generating a response according to the progress of the scenario.

[1680] (Claim 3)

[1681] 10. The system of claim 1, wherein the server further comprises means for checking the authentication information against a database.

[1682] "Example 1"

[1683] (Claim 1)

[1684] a server means for providing a role-play simulation for users to learn customer service skills;

[1685] A means for the terminal to perform login authentication of a user;

[1686] a means by which the server verifies user authentication;

[1687] a means for the terminal to display scenario options to the user;

[1688] means for the server to prepare data based on the selected scenario;

[1689] a means for the server to parse the user's input using a natural language processing library;

[1690] A means for the server to generate a response for the customer using the generative AI model;

[1691] A means for the terminal to display a screen for starting a scenario;

[1692] A means for the server to progress the scenario and respond to input from the user;

[1693] a means for the server to generate feedback using the analytical model after the scenario is completed;

[1694] means for the terminal to display feedback to the user;

[1695] A system including:

[1696] (Claim 2)

[1697] 2. The system of claim 1, wherein the server further comprises means for analyzing a user's input and generating a response according to the progress of the scenario.

[1698] (Claim 3)

[1699] 10. The system of claim 1, wherein the server further comprises means for checking the authentication information against a database.

[1700] "Application Example 1"

[1701] (Claim 1)

[1702] an information processing device that provides a role-play simulation for a user to learn customer service skills;

[1703] A means for the operation terminal to perform login authentication of a user;

[1704] means for verifying user authentication by the information processing device;

[1705] a means for displaying scenario options to a user on an operation terminal;

[1706] A means for preparing data based on the scenario selected by the information processing device;

[1707] a means for displaying a scenario start screen on the operation terminal;

[1708] a means for the information processing device to progress through a scenario and respond to input from a user;

[1709] a means for generating feedback after the scenario is completed by the information processing device;

[1710] a means for the operation terminal to display feedback to the user;

[1711] a means for using a generative AI model to generate feedback according to a user's response input from an operation terminal;

[1712] a means for inputting a prompt to the generative AI model;

[1713] A system including:

[1714] (Claim 2)

[1715] 2. The system according to claim 1, wherein the information processing device further comprises means for analyzing a user's input and generating a response according to the progress of the scenario.

[1716] (Claim 3)

[1717] 10. The system of claim 1, wherein the information processing device further comprises means for comparing the authentication information with a database.

[1718] "Example 2: Combining Emotion Engines"

[1719] (Claim 1)

[1720] a computer means for providing a role-play simulation for a user to learn customer service skills;

[1721] A means for the terminal to perform login authentication of a user;

[1722] a means for the computer to verify user authentication;

[1723] a means for the terminal to display scenario options to the user;

[1724] means for the computer to prepare data based on the selected scenario;

[1725] A means for the terminal to display a screen for starting a scenario;

[1726] a means for the computer to progress through the scenario and respond to input from the user;

[1727] means for the terminal to send user input to the emotion engine;

[1728] means for the emotion engine to analyze user input and generate emotion data;

[1729] A means for a computer to generate a response based on the emotion analysis result;

[1730] a means for the computer to generate feedback after the scenario is completed; and

[1731] means for the terminal to display feedback to the user;

[1732] A system including:

[1733] (Claim 2)

[1734] 10. The system of claim 1, further comprising means for the computer to analyze user input and generate responses according to the progress of the scenario.

[1735] (Claim 3)

[1736] 10. The system of claim 1, further comprising means for the computer to check the authentication information against a database.

[1737] "Application example 2 when combining emotion engines"

[1738] (Claim 1)

[1739] a server means for providing a role-play simulation for users to learn customer service skills;

[1740] A means for the terminal to perform login authentication of a user;

[1741] a means by which the server verifies user authentication;

[1742] a means for the terminal to display scenario options to the user;

[1743] means for the server to prepare data based on the selected scenario;

[1744] A means for the terminal to display a screen for starting a scenario;

[1745] A means for the server to progress the scenario and respond to input from the user;

[1746] an emotion engine means for the server to recognize the emotion of the visitor in real time;

[1747] A means for the server to adjust the speed and content of explanation according to the recognized emotion;

[1748] A means for the server to generate feedback after the scenario is completed;

[1749] means for the terminal to display feedback to the user;

[1750] A system including:

[1751] (Claim 2)

[1752] 2. The system of claim 1, wherein the server further comprises means for analyzing a user's input and generating a response according to the progress of the scenario.

[1753] (Claim 3)

[1754] 10. The system of claim 1, wherein the server further comprises means for checking the authentication information against a database. [Explanation of symbols]

[1755] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a server means for providing a role-play simulation for users to learn customer service skills; A means for the terminal to perform login authentication of a user; a means by which the server verifies user authentication; a means for the terminal to display scenario options to the user; means for the server to prepare data based on the selected scenario; A means for the terminal to display a screen for starting a scenario; A means for the server to progress the scenario and respond to input from the user; A means for the server to generate feedback after the scenario is completed; means for the terminal to display feedback to the user; A system including:

2. 2. The system according to claim 1, wherein the server further comprises means for analyzing a user's input and generating a response according to the progress of the scenario.

3. 10. The system of claim 1, further comprising means for the server to check the authentication information against a database.

Images