system

Abstract

We provide the system. [Solution] A means for generating multiple response scenarios using generative artificial intelligence, A means for analyzing response data received from users and creating appropriate responses using generative artificial intelligence, A means for conducting training sessions for users and evaluating the results of those sessions, A means of providing feedback to users based on the evaluation results, A system that includes means for retaining and managing training history and evaluation information.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional training methods aimed at improving customer service skills in customer service are difficult to provide practical training that conforms to real-world scenarios and are difficult to provide effective feedback. In addition, there is a problem that it is costly to achieve consistent quality improvement. Furthermore, if flexible training that can handle different customer attributes is not provided, the skills of the staff will only be improved locally and limitedly, which is not sufficient to improve the service quality of the entire organization.

Means for Solving the Problems

[0005] We have developed a system that utilizes generative artificial intelligence to generate various customer interaction scenarios and provide users with interactive training based on these scenarios. This system evaluates users' interactions in real time and provides specific feedback, thereby promoting individual skill improvement. Furthermore, by centrally managing training history and evaluation information, and allowing users to visualize their learning progress, it provides an environment where skills can be improved efficiently and effectively. In addition, by simulating customer roles with various attributes using generative artificial intelligence, it enables training that can handle diverse situations, contributing to the improvement of service quality across the entire organization.

[0006] "Generative artificial intelligence" is a type of artificial intelligence that has the ability to learn from large amounts of data and generate new information or responses based on that data.

[0007] A "customer interaction scenario" is a set of dialogue formats designed to recreate a conversation between a user and a virtual customer under specific circumstances.

[0008] "Response data" refers to the information and statements that a user enters during an interaction.

[0009] A "training session" is a portion of the practice time in which users practice virtual customer interactions based on scenarios to improve their skills.

[0010] "Feedback" refers to information provided by users that includes evaluations of their actions and specific suggestions for improvement.

[0011] "Training history" refers to the records and historical information of training sessions that a user has taken so far.

[0012] "Evaluation information" refers to indicators and data that show the evaluation results of user responses.

[0013] The "customer role" refers to the role of a virtual customer played by the generative artificial intelligence during training.

[0014] "Interaction" refers to two-way communication or dialogue between a user and a system. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a RAM (Random Access Memory) with a reference number is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a storage with a reference number is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a training system aimed at effectively improving customer service skills in the service industry. The system consists of a server, terminals, and users, and utilizes a generative artificial intelligence model to provide diverse customer service scenarios.

[0037] The server runs generative artificial intelligence to generate various interaction scenarios. These scenarios have different dialogues based on various customer attributes and needs. The server sends these scenarios to the terminal, allowing the user to select one. The user can then select a scenario that suits their skill level and learning objectives via the terminal and begin training.

[0038] During training sessions, users interact with a customer role played by AI through their device. User input data is sent to a server in real time and analyzed by generative artificial intelligence. Based on this analysis, the server generates an appropriate response and sends it to the device. This allows users to experience virtual customer service and develop skills relevant to actual work.

[0039] Once the training is complete, the server evaluates the user's responses and provides specific feedback through the terminal. This feedback includes aspects such as the accuracy of the response, consideration for customer emotions, and areas for improvement, helping the user improve their skills. Furthermore, the server stores training history and evaluation information in a database, allowing users to track their learning progress.

[0040] As a concrete example, when a new staff member is being trained to deal with a "highly dissatisfied customer," the generative artificial intelligence sets up a virtual customer with specific reasons for dissatisfaction and provides the user with a role-playing scenario. The user attempts to respond from their terminal, and the server evaluates their response. Through this process, the user can directly and quickly improve their problem-solving abilities. In this way, the system enables efficient skill improvement and contributes to improving the quality of customer service.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The server activates a generative artificial intelligence model to generate various customer interaction scenarios. This includes setting up dialogues based on different customer attributes and their respective situations.

[0044] Step 2:

[0045] The server sends the generated scenarios to the terminal as a list. The terminal displays this list to the user, allowing the user to select the scenario they want to train.

[0046] Step 3:

[0047] The user selects a scenario through their device. The selection information is sent from the device to the server, which then configures the AI ​​character based on the selected scenario.

[0048] Step 4:

[0049] The training session begins, and the user interacts with a customer role played by AI via the device. This interaction is conducted via voice or text input.

[0050] Step 5:

[0051] User input data is sent from the terminal to the server in real time. The server analyzes the input and uses an AI model to generate appropriate responses as a customer.

[0052] Step 6:

[0053] The server sends the generated response to the terminal. The terminal then displays or audibly presents this response to the user, allowing the conversation to continue.

[0054] Step 7:

[0055] After the session ends, the server evaluates the user's response. This evaluation includes metrics such as accuracy, level of consideration, and room for improvement.

[0056] Step 8:

[0057] Based on the evaluation results, the server generates feedback for the user. This feedback is provided to the user via the terminal.

[0058] Step 9:

[0059] The server records training history and evaluation information in a database. Users can access this information from their terminals and use it as a guide for self-study.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] In today's service industry, the quality of customer service is a crucial factor directly linked to a company's success. However, traditional training methods have made it difficult to customize training content to the individual needs and skill levels of employees. Furthermore, there is a challenge in developing realistic customer service skills without actual interaction with customers. In addition, there is a lack of mechanisms for employees to monitor their own progress and continuously improve their skills.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for generating dialogue scenarios with different conditions using generative artificial intelligence, means for analyzing data received from an information processing device and generating the optimal dialogue using generative artificial intelligence, and means for conducting training sessions for users and determining the results of those sessions. This enables customized training tailored to individual users, progress tracking, and practical and realistic customer service training.

[0065] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new data and information based on given input. This technology allows for the automatic creation of diverse scenarios and responses.

[0066] "Conditions" refer to the various elements and circumstances considered when forming a particular scenario or response, which allows the resulting content to have different characteristics.

[0067] An "information processing device" refers to equipment that receives data and appropriately analyzes and processes it, and mainly includes hardware such as computers and servers.

[0068] A "dialogue scenario" refers to a series of dialogues designed with a specific purpose in mind, providing users with realistic training through content that reflects different conditions.

[0069] "Progress" refers to the level of improvement in skills and abilities that users have achieved through training, and is evaluated by comparing it to their past state.

[0070] "Judgment" refers to the process by which an information processing device evaluates the results of a training session and analyzes the findings, clearly indicating areas for improvement and the degree of achievement.

[0071] To implement this invention, it is necessary to configure a system using a generative artificial intelligence model. The server is responsible for driving this generative AI model and generating various response scenarios. Specifically, the server takes prompt text as input, and the AI ​​model generates a variety of scenarios necessary for customer service. An example of such prompt text would be, "Generate scenarios for customer service training in the service industry. Please include customer attributes and needs." The generated scenarios reflect different customer attributes and situations, and aim to improve the user's ability to handle various cases.

[0072] The terminal is a device that receives scenarios sent from the server and provides them to the user. A list of selectable scenarios is displayed on the terminal, and the user can choose a scenario according to their learning objectives.

[0073] The user interacts with a virtual customer, played by AI, via their device. The user's responses are sent to the server in real time, where a generative AI model analyzes them and generates the next response. Through this process, the user can experience virtual customer service. Once training is complete, the server evaluates the user's responses and provides specific feedback via the device. This feedback includes aspects such as the accuracy of the response, consideration of customer emotions, and areas for improvement.

[0074] A concrete example is a scenario where new staff members undergo training to handle "highly dissatisfied customers." In this scenario, generative artificial intelligence sets up a virtual customer with specific reasons for dissatisfaction and provides the user with role-playing through dialogue. Through this process, users can improve their problem-solving abilities in a practical and rapid manner. This system is expected to efficiently improve skills in the service industry and directly contribute to improving the quality of work.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The server supplies prompt text as input to the AI ​​model. This input allows the AI ​​model to generate conversational scenarios based on customer attributes and needs. The generated scenarios are saved as server output and ready to be provided to the user later.

[0078] Step 2:

[0079] The server sends the generated scenarios to the terminal. On the terminal, multiple scenarios are displayed in a list format, allowing the user to select one. Through this data transfer, the terminal plays the role of providing the user with choices.

[0080] Step 3:

[0081] The user uses their device to select a scenario that suits their needs and learning goals, and then begins training. The selected scenario is notified from the device to the server based on the user's selection, and the server receives a signal to begin training.

[0082] Step 4:

[0083] Based on scenario information from the server, the terminal presents the user with the first question from a virtual customer played by AI. Once the user's input is received by the terminal, it is sent to the server.

[0084] Step 5:

[0085] The server receives user input and analyzes it using generative artificial intelligence. Based on this analysis, it generates the next appropriate customer response. The generated response is sent to the terminal as server output, and the interaction with the user continues.

[0086] Step 6:

[0087] After training is complete, the server evaluates all interactions. This evaluation includes the appropriateness of the interaction and the level of understanding of customer sentiment. The evaluation results are sent from the server to the terminal, which then presents them to the user as feedback.

[0088] Step 7:

[0089] The server stores data obtained during training sessions in a database. This includes the user's interaction history and evaluation results. The stored data allows users to check their skill progress from their devices. Access to this information facilitates continuous skill improvement.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] Improving customer service skills is crucial in customer service roles, but there is a challenge in conducting realistic training that closely reflects actual work situations. Furthermore, developing the ability to respond flexibly to different customer attributes requires the creation of diverse scenarios, but efficiently implementing these is difficult.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes means for generating multiple response scenarios using generative artificial intelligence, means for providing virtual dialogues to the user in real time via a visual interface device, and means for providing real-time feedback via the visual interface device. This enables realistic training in an actual work environment and rapid skill improvement of staff.

[0095] "Generative artificial intelligence" is a technology that automatically generates diverse dialogue scenarios and creates appropriate responses through interaction with the user.

[0096] "Response data" refers to information received from the user, which forms the basis for generative artificial intelligence to analyze and generate appropriate dialogue.

[0097] A "training session" is a process in which a user interacts with generative artificial intelligence to improve their skills.

[0098] "Evaluation results" refer to data used to assess a user's communication skills during a training session and to provide feedback on those skills.

[0099] A "visual interface device" is a device that allows users to visually experience virtual interactions, and in most cases refers to devices such as smart glasses.

[0100] "Real-time feedback" is feedback generated instantly in response to user actions and responses, providing immediate points for improvement to enhance skills.

[0101] The system for implementing this invention mainly consists of a server, a visual interface device (e.g., smart glasses), and a user terminal. The server is responsible for generating multiple interaction scenarios using a generative AI model. This AI model automatically generates diverse customer scenarios, creating an environment in which users can interact with virtual customers with different attributes.

[0102] The visual interface device is designed to provide users with virtual interactions in real time, allowing them to communicate with the system. The user's spoken content and responses are transmitted to a server via the terminal, which analyzes them using a generative AI model. The server then generates appropriate responses and returns them in real time, providing a realistic customer service training environment.

[0103] Feedback is provided immediately through a visual interface device, allowing users to make corrections and improvements in real time. After the training session ends, the server evaluates the user's responses and sends feedback based on the evaluation results to the terminal. This feedback includes accuracy of responses and areas for improvement.

[0104] One concrete example is a scenario in which department store sales staff use smart glasses to interact with virtual customers with different personalities and needs. During this training, users can constantly try out appropriate responses and hone their skills. An example of a prompt might be, "The customer is having trouble with the fit of their jeans. How would the customer express their dissatisfaction?"

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The server generates multiple interaction scenarios using generative artificial intelligence. It receives diverse customer attribute and needs information as input and analyzes it using a generative AI model. As output, it generates unique dialogue scenarios, which are then prepared for the user to select in the next step.

[0108] Step 2:

[0109] The terminal presents the user with response scenarios received from the server. The input is scenario data from the server, and the output is a scenario selection screen on the user interface. The user selects a scenario appropriate to their level through the terminal and begins training.

[0110] Step 3:

[0111] The user interacts with a virtual customer using a visual interface device. Input consists of audio and text data of the user's responses, which the visual interface transmits to the server in real time. Output is recorded as the user's response and sent to the next analysis step.

[0112] Step 4:

[0113] The server analyzes response data received from the user. The input is real-time text or audio data. Data analysis is performed, and an optimized response is generated using generative artificial intelligence. Prompts are used to improve the dialogue generation. The output is returned to the user as the next message from a virtual customer.

[0114] Step 5:

[0115] The visual interface device provides the user with generated responses in real time and allows them to evaluate the responses. The input is the response from the server, and the output is feedback information that the user sees. Furthermore, suggested areas for improvement are presented, allowing the user to improve the response on the spot.

[0116] Step 6:

[0117] Once the training session is complete, the server comprehensively evaluates the overall interaction. The input consists of all responses and interaction data recorded during training. Based on the evaluation criteria, the server performs analysis and sends detailed evaluation results and feedback to the terminal as output.

[0118] Step 7:

[0119] The terminal displays detailed evaluation results sent from the server to the user. The input is evaluation data from the server, and the output is a feedback screen that the user reviews. This includes accuracy of the response, consideration for customer feelings, and areas for improvement, which the user can review to improve their skills.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] This invention provides a training system for improving customer service skills in the service industry, using generative artificial intelligence and an emotion engine. The system consists of a server, terminals, and a user, and simulates actual customer interactions while providing feedback that takes into account the user's emotional state.

[0122] The server activates generative artificial intelligence to generate a variety of interaction scenarios. These scenarios include dialogue settings that simulate different customer attributes and situations. These scenarios are presented to the user via a terminal, and the user is configured to select a scenario that suits their training objectives.

[0123] Once the user selects a scenario, a training session begins. During the session, the device utilizes an emotion engine to recognize the user's emotions in real time from their facial expressions and tone of voice. This allows the server to understand the user's emotional state and provide appropriate responses and follow-up. Responses are generated by generative artificial intelligence, providing the user with the most optimal interaction.

[0124] This real-time emotion recognition further improves training efficiency, allowing users to manage emotional stress while enhancing their customer service skills in more realistic situations. Additionally, the emotion data captured by the emotion engine is sent to a server and used to evaluate the user's response tendencies and emotional response capabilities.

[0125] Once training is complete, the server comprehensively evaluates the user's interactions and provides feedback based on the results. This feedback includes specific advice on emotional handling and the quality of interactions. Furthermore, in addition to training history and evaluation information, the server records emotional data in a database, providing information that helps track the user's learning progress and contributes to long-term skill improvement.

[0126] As a concrete example, in training on effective complaint handling, the generative artificial intelligence sets up a customer role with volatile emotions, and the user attempts to respond to this situation. The emotion engine analyzes the subtle emotional changes the user exhibits in real time, and appropriate advice based on this information is provided during and after the training. This allows the user to improve not only their technical skills but also their emotional response capabilities. In this way, the system makes a significant contribution to improving the quality of service in the hospitality industry.

[0127] The following describes the processing flow.

[0128] Step 1:

[0129] The server activates generative artificial intelligence to design and generate various customer interaction scenarios. These scenarios incorporate different customer attributes and create a list for the user to select from.

[0130] Step 2:

[0131] The terminal displays a list of scenarios sent from the server to the user. The user selects a scenario that suits their training needs and sends that selection from the terminal to the server.

[0132] Step 3:

[0133] The server configures the AI ​​character and interaction environment based on the selected scenario, and prepares to start the training session.

[0134] Step 4:

[0135] When a training session begins, the device activates its emotion engine, analyzing the user's facial expressions and voice in real time to acquire emotion data. This data is immediately sent to the server.

[0136] Step 5:

[0137] The user interacts with an AI-generated customer through their device. Based on emotional data provided by the emotion engine, the server generates the optimal response each time and adjusts the feedback provided to the user.

[0138] Step 6:

[0139] The server links user responses with emotional data, generates an AI response at the appropriate time, and sends it to the device for display to the user. This allows the user to continue interacting while receiving continuous feedback.

[0140] Step 7:

[0141] Once the training session is complete, the server comprehensively evaluates the user's interaction and emotional data to generate personalized feedback.

[0142] Step 8:

[0143] Feedback is provided to users through their devices and includes suggestions for improving customer service and advice on emotional responses.

[0144] Step 9:

[0145] The server records training history, evaluation information, and sentiment data in a database. Users can view this information on their devices and use it to improve their skills in the future.

[0146] (Example 2)

[0147] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0148] While customer service skills are becoming increasingly important in the service industry, traditional training methods struggle to improve these skills in realistic situations. In particular, there is a growing need to enhance the ability to respond to users' emotional states, but effective methods for achieving this are lacking. To address these issues, there is a need for the development of interactive training systems that analyze users' emotional states and incorporate the results.

[0149] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0150] In this invention, the server includes means for generating multiple dialogue scenarios using generative artificial intelligence, means for analyzing response data received from the user and creating an appropriate response using generative artificial intelligence, and means for analyzing the user's emotional state in real time using an emotion analysis engine and generating feedback based on that data. This makes it possible to provide a realistic simulation environment that takes the user's emotional state into account and to efficiently improve customer service skills.

[0151] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to automatically generate text, such as response scenarios, based on given prompts.

[0152] A "dialogue scenario" is a practice scenario that includes hypothetical conversation content and is used to simulate different customer attributes and situations.

[0153] "Response data" refers to the collection of reactions and input information that a user provides during an interaction.

[0154] An "emotion analysis engine" is a technology that evaluates and detects a user's emotional state in real time from data such as facial expressions and voice tone.

[0155] "Feedback" refers to information that includes evaluations and suggestions for improvement regarding the user's interactions, and is provided as part of training.

[0156] An "interface" is a means or method that provides a way for a user and a system to exchange information.

[0157] This invention is a training system for improving customer service skills in the service industry. The following describes a specific embodiment for carrying out the invention.

[0158] The server runs a generative artificial intelligence model on industry-standard computing equipment. This model utilizes generative AI models such as OpenAI's GPT series to generate customer service dialogue scenarios. An example prompt statement could be, "Generate dialogue scenarios that consider different customer attributes and situations." These generated scenarios include a variety of customer attributes and response patterns.

[0159] The terminal is a device that functions as a user interface, providing a list of scenarios for the user to select. The terminal has a touchscreen display and audio input / output devices to accept user input. Based on the selected scenario, the terminal activates an emotion analysis engine and analyzes the user's emotional state in real time. This analysis is performed using EmotionAPI and other emotion recognition software. Based on data obtained from the user's facial expressions and tone of voice, the emotional state is quantified and sent to the server.

[0160] The server uses a generative AI model to generate optimal responses and feedback based on received sentiment data and user response data. This feedback includes specific advice on areas for improvement and skills to strengthen. This feedback, delivered through the user interface, allows users to improve their skills step-by-step and practically.

[0161] As a concrete example, during complaint handling training, generative artificial intelligence sets up a simulation role using the prompt, "Generate a conversation scenario that guides an angry customer to calm down." As the user attempts to handle the situation, the device recognizes emotions and continuously provides appropriate advice. This system allows users to efficiently improve their practical skills in customer service.

[0162] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0163] Step 1: Scenario Generation

[0164] The server activates a generative AI model and, using the configured prompt text as input, generates a variety of interaction scenarios. These scenarios include different customer attributes and situations. The output is a virtual dialogue scenario, which is saved in text format.

[0165] Step 2: Presenting the Scenario

[0166] The terminal receives scenarios output from the server and presents them to the user through a user interface. Specifically, it displays a list of scenarios on the terminal's display and allows selection via touch operation. The input is a list of scenarios, and the output is the selected scenario.

[0167] Step 3: Analysis of emotional data

[0168] The device activates an emotion analysis engine based on a scenario selected by the user. It analyzes the user's facial expressions and voice tone in real time and acquires this information as digital data. The input is the user's voice and video, and the output is quantified emotion data.

[0169] Step 4: Generating response feedback

[0170] The server receives emotion data and user responses from the terminal as input and uses a generative AI model to generate optimal feedback. The generated feedback includes areas for improvement and hints for better responses. The output is feedback in text format.

[0171] Step 5: Provide feedback

[0172] The terminal provides the user with feedback obtained from the server. Specifically, it can display advice on the terminal screen and also provide voice guidance. The input is text data of the feedback, and the output is the presentation of visual or auditory information to the user.

[0173] In this way, the system provides users with training in a realistic environment and helps improve their customer service skills based on emotion recognition.

[0174] (Application Example 2)

[0175] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0176] In the service industry, a challenge is providing real-time feedback to efficiently improve employees' customer service skills. Existing training systems lack the ability to handle diverse customer attributes and improve emotional care skills, and there is a lack of appropriate feedback that utilizes emotional information.

[0177] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0178] In this invention, the server includes means for generating multiple response scenarios using generative artificial intelligence, means for recognizing the user's emotional state in real time and optimizing the interaction using that information, and means for providing real-time feedback through an adaptive human interface. This enables users to efficiently improve their customer service skills.

[0179] "Generative artificial intelligence" is an artificial intelligence technology that generates responses based on user interaction and provides various response scenarios.

[0180] A "customer service scenario" is a hypothetical response case designed to simulate actual customer service situations and improve user skills.

[0181] "Emotional state" refers to the psychological and emotional condition that can be determined from the user's facial expressions and voice.

[0182] "Interaction" refers to the dynamic, two-way exchange that takes place between the user and the system.

[0183] An "adaptive human interface" is a point of contact between humans and machines that dynamically changes according to the user's situation to provide optimal feedback.

[0184] "Feedback" is a means of providing information to improve user performance based on the results of training sessions.

[0185] The system for realizing this invention primarily consists of a server, a terminal, and a user. This system is designed to improve customer service skills in the service industry. The server uses generative artificial intelligence to generate diverse response scenarios and provides them to the user. The user can access these scenarios through the terminal and implement appropriate responses.

[0186] The device uses an emotion engine to analyze the user's voice tone and facial expressions in real time. This emotion engine accurately captures the user's emotional state and transmits it to the server. This allows the server to generate responses and feedback tailored to the user's emotional state, providing a more realistic training environment. The server also maintains the user's training history and evaluation information to support long-term skill improvement.

[0187] As a concrete example, consider a scenario where a cafe employee practices handling customer complaints due to order errors. The server generates a scenario titled "Generating AI Model, Prompt Sentence," featuring a customer with rapidly changing emotions. The user selects this scenario and responds through interaction with the virtual customer. During this process, the terminal detects the user's emotional changes and provides appropriate feedback. For example, a prompt sentence such as "Please tell me how to respond when a customer is served the wrong drink" could be used. In this way, the system of the present invention enhances the user's ability to respond to emotional issues and helps them acquire better customer service skills.

[0188] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0189] Step 1:

[0190] The server generates diverse interaction scenarios using generative artificial intelligence. The server receives user training objectives and configuration parameters as input. Based on these inputs, the generative AI model generates scenarios with appropriate prompts and different customer attributes. The output consists of multiple virtual scenarios that the user can select from.

[0191] Step 2:

[0192] The user uses a terminal to select a response scenario provided by the server. During this selection process, the user interface displays an overview of each scenario. The terminal receives the user's selection information, and the output is a message indicating that the selected scenario is ready to begin.

[0193] Step 3:

[0194] The device uses an emotion engine to analyze the user's facial expressions and voice in real time. The input to the device consists of the user's raw voice data and facial image data. This data is processed using an emotion analysis algorithm to identify the user's emotional state. The output is the analyzed emotion data.

[0195] Step 4:

[0196] The server receives the analyzed emotion data and generates optimal feedback tailored to the user's emotional state. The server receives the result of the emotion analysis as input. Based on this data, the server utilizes a generation AI model to generate appropriate responses and follow-ups for the user. The output is a feedback message.

[0197] Step 5:

[0198] Feedback is provided to the user in real time via the device. Through this feedback, the user has the opportunity to adjust their response methods. The input to the device is a feedback message sent from the server, and the output is improvement guidelines provided to the user.

[0199] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0200] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0201] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0202] [Second Embodiment]

[0203] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0204] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0205] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0206] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0207] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0208] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0209] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0210] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0211] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0212] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0213] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0214] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0215] This invention is a training system aimed at effectively improving customer service skills in the service industry. The system consists of a server, terminals, and users, and utilizes a generative artificial intelligence model to provide diverse customer service scenarios.

[0216] The server runs generative artificial intelligence to generate various interaction scenarios. These scenarios have different dialogues based on various customer attributes and needs. The server sends these scenarios to the terminal, allowing the user to select one. The user can then select a scenario that suits their skill level and learning objectives via the terminal and begin training.

[0217] During training sessions, users interact with a customer role played by AI through their device. User input data is sent to a server in real time and analyzed by generative artificial intelligence. Based on this analysis, the server generates an appropriate response and sends it to the device. This allows users to experience virtual customer service and develop skills relevant to actual work.

[0218] Once the training is complete, the server evaluates the user's responses and provides specific feedback through the terminal. This feedback includes aspects such as the accuracy of the response, consideration for customer emotions, and areas for improvement, helping the user improve their skills. Furthermore, the server stores training history and evaluation information in a database, allowing users to track their learning progress.

[0219] As a concrete example, when a new staff member is being trained to deal with a "highly dissatisfied customer," the generative artificial intelligence sets up a virtual customer with specific reasons for dissatisfaction and provides the user with a role-playing scenario. The user attempts to respond from their terminal, and the server evaluates their response. Through this process, the user can directly and quickly improve their problem-solving abilities. In this way, the system enables efficient skill improvement and contributes to improving the quality of customer service.

[0220] The following describes the processing flow.

[0221] Step 1:

[0222] The server activates a generative artificial intelligence model to generate various customer interaction scenarios. This includes setting up dialogues based on different customer attributes and their respective situations.

[0223] Step 2:

[0224] The server sends the generated scenarios to the terminal as a list. The terminal displays this list to the user, allowing the user to select the scenario they want to train.

[0225] Step 3:

[0226] The user selects a scenario through their device. The selection information is sent from the device to the server, which then configures the AI ​​character based on the selected scenario.

[0227] Step 4:

[0228] The training session begins, and the user interacts with a customer role played by AI via the device. This interaction is conducted via voice or text input.

[0229] Step 5:

[0230] User input data is sent from the terminal to the server in real time. The server analyzes the input and uses an AI model to generate appropriate responses as a customer.

[0231] Step 6:

[0232] The server sends the generated response to the terminal. The terminal then displays or audibly presents this response to the user, allowing the conversation to continue.

[0233] Step 7:

[0234] After the session ends, the server evaluates the user's response. This evaluation includes metrics such as accuracy, level of consideration, and room for improvement.

[0235] Step 8:

[0236] Based on the evaluation results, the server generates feedback for the user. This feedback is provided to the user via the terminal.

[0237] Step 9:

[0238] The server records training history and evaluation information in a database. Users can access this information from their terminals and use it as a guide for self-study.

[0239] (Example 1)

[0240] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0241] In today's service industry, the quality of customer service is a crucial factor directly linked to a company's success. However, traditional training methods have made it difficult to customize training content to the individual needs and skill levels of employees. Furthermore, there is a challenge in developing realistic customer service skills without actual interaction with customers. In addition, there is a lack of mechanisms for employees to monitor their own progress and continuously improve their skills.

[0242] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0243] In this invention, the server includes means for generating dialogue scenarios with different conditions using generative artificial intelligence, means for analyzing data received from an information processing device and generating the optimal dialogue using generative artificial intelligence, and means for conducting training sessions for users and determining the results of those sessions. This enables customized training tailored to individual users, progress tracking, and practical and realistic customer service training.

[0244] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new data and information based on given input. This technology allows for the automatic creation of diverse scenarios and responses.

[0245] "Conditions" refer to the various elements and circumstances considered when forming a particular scenario or response, which allows the resulting content to have different characteristics.

[0246] An "information processing device" refers to equipment that receives data and appropriately analyzes and processes it, and mainly includes hardware such as computers and servers.

[0247] A "dialogue scenario" refers to a series of dialogues designed with a specific purpose in mind, providing users with realistic training through content that reflects different conditions.

[0248] "Progress" refers to the level of improvement in skills and abilities that users have achieved through training, and is evaluated by comparing it to their past state.

[0249] "Judgment" refers to the process by which an information processing device evaluates the results of a training session and analyzes the findings, clearly indicating areas for improvement and the degree of achievement.

[0250] To implement this invention, it is necessary to configure a system using a generative artificial intelligence model. The server is responsible for driving this generative AI model and generating various response scenarios. Specifically, the server takes prompt text as input, and the AI ​​model generates a variety of scenarios necessary for customer service. An example of such prompt text would be, "Generate scenarios for customer service training in the service industry. Please include customer attributes and needs." The generated scenarios reflect different customer attributes and situations, and aim to improve the user's ability to handle various cases.

[0251] The terminal is a device that receives scenarios sent from the server and provides them to the user. A list of selectable scenarios is displayed on the terminal, and the user can choose a scenario according to their learning objectives.

[0252] The user interacts with a virtual customer, played by AI, via their device. The user's responses are sent to the server in real time, where a generative AI model analyzes them and generates the next response. Through this process, the user can experience virtual customer service. Once training is complete, the server evaluates the user's responses and provides specific feedback via the device. This feedback includes aspects such as the accuracy of the response, consideration of customer emotions, and areas for improvement.

[0253] A concrete example is a scenario where new staff members undergo training to handle "highly dissatisfied customers." In this scenario, generative artificial intelligence sets up a virtual customer with specific reasons for dissatisfaction and provides the user with role-playing through dialogue. Through this process, users can improve their problem-solving abilities in a practical and rapid manner. This system is expected to efficiently improve skills in the service industry and directly contribute to improving the quality of work.

[0254] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0255] Step 1:

[0256] The server supplies prompt text as input to the AI ​​model. This input allows the AI ​​model to generate conversational scenarios based on customer attributes and needs. The generated scenarios are saved as server output and ready to be provided to the user later.

[0257] Step 2:

[0258] The server sends the generated scenarios to the terminal. On the terminal, multiple scenarios are displayed in a list format, allowing the user to select one. Through this data transfer, the terminal plays the role of providing the user with choices.

[0259] Step 3:

[0260] The user uses their device to select a scenario that suits their needs and learning goals, and then begins training. The selected scenario is notified from the device to the server based on the user's selection, and the server receives a signal to begin training.

[0261] Step 4:

[0262] Based on scenario information from the server, the terminal presents the user with the first question from a virtual customer played by AI. Once the user's input is received by the terminal, it is sent to the server.

[0263] Step 5:

[0264] The server receives user input and analyzes it using generative artificial intelligence. Based on this analysis, it generates the next appropriate customer response. The generated response is sent to the terminal as server output, and the interaction with the user continues.

[0265] Step 6:

[0266] After training is complete, the server evaluates all interactions. This evaluation includes the appropriateness of the interaction and the level of understanding of customer sentiment. The evaluation results are sent from the server to the terminal, which then presents them to the user as feedback.

[0267] Step 7:

[0268] The server stores data obtained during training sessions in a database. This includes the user's interaction history and evaluation results. The stored data allows users to check their skill progress from their devices. Access to this information facilitates continuous skill improvement.

[0269] (Application Example 1)

[0270] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0271] Improving customer service skills is crucial in customer service roles, but there is a challenge in conducting realistic training that closely reflects actual work situations. Furthermore, developing the ability to respond flexibly to different customer attributes requires the creation of diverse scenarios, but efficiently implementing these is difficult.

[0272] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0273] In this invention, the server includes means for generating multiple response scenarios using generative artificial intelligence, means for providing virtual dialogues to the user in real time via a visual interface device, and means for providing real-time feedback via the visual interface device. This enables realistic training in an actual work environment and rapid skill improvement of staff.

[0274] "Generative artificial intelligence" is a technology that automatically generates diverse dialogue scenarios and creates appropriate responses through interaction with the user.

[0275] "Response data" refers to information received from the user, which forms the basis for generative artificial intelligence to analyze and generate appropriate dialogue.

[0276] A "training session" is a process in which a user interacts with generative artificial intelligence to improve their skills.

[0277] "Evaluation results" refer to data used to assess a user's communication skills during a training session and to provide feedback on those skills.

[0278] A "visual interface device" is a device that allows users to visually experience virtual interactions, and in most cases refers to devices such as smart glasses.

[0279] "Real-time feedback" is feedback generated instantly in response to user actions and responses, providing immediate points for improvement to enhance skills.

[0280] The system for implementing this invention mainly consists of a server, a visual interface device (e.g., smart glasses), and a user terminal. The server is responsible for generating multiple interaction scenarios using a generative AI model. This AI model automatically generates diverse customer scenarios, creating an environment in which users can interact with virtual customers with different attributes.

[0281] The visual interface device is designed to provide users with virtual interactions in real time, allowing them to communicate with the system. The user's spoken content and responses are transmitted to a server via the terminal, which analyzes them using a generative AI model. The server then generates appropriate responses and returns them in real time, providing a realistic customer service training environment.

[0282] Feedback is provided immediately through a visual interface device, allowing users to make corrections and improvements in real time. After the training session ends, the server evaluates the user's responses and sends feedback based on the evaluation results to the terminal. This feedback includes accuracy of responses and areas for improvement.

[0283] As a specific example, there is a scenario where a customer service staff in a department store uses smart glasses to interact with virtual customers with different personalities and needs. During this training, the user can always try appropriate response methods and hone their skills. As an example of a prompt sentence, "The customer is having trouble fitting into the jeans. How might the customer express dissatisfaction?" can be considered.

[0284] The flow of the specific process in Application Example 1 will be described with reference to FIG. 12.

[0285] Step 1:

[0286] The server generates a plurality of response scenarios using generative artificial intelligence. As input, it receives diverse customer attribute and need information, and analyzes this using a generative AI model. As output, it generates different dialogue scenarios and prepares these in a state where the user can select them in the next step.

[0287] Step 2:

[0288] The terminal presents the response scenario received from the server to the user. The input is the scenario data from the server, and the output is the scenario selection screen on the user interface. The user selects a scenario appropriate to their level through the terminal and starts the training.

[0289] Step 3:

[0290] The user interacts with the virtual customer using a visual interface device. The input is the voice or text data of the user's response, which the visual interface transmits to the server in real time. The output is recorded as the user's response and sent to the next analysis step.

[0291] Step 4:

[0292] The server analyzes response data received from the user. The input is real-time text or audio data. Data analysis is performed, and an optimized response is generated using generative artificial intelligence. Prompts are used to improve the dialogue generation. The output is returned to the user as the next message from a virtual customer.

[0293] Step 5:

[0294] The visual interface device provides the user with generated responses in real time and allows them to evaluate the responses. The input is the response from the server, and the output is feedback information that the user sees. Furthermore, suggested areas for improvement are presented, allowing the user to improve the response on the spot.

[0295] Step 6:

[0296] Once the training session is complete, the server comprehensively evaluates the overall interaction. The input consists of all responses and interaction data recorded during training. Based on the evaluation criteria, the server performs analysis and sends detailed evaluation results and feedback to the terminal as output.

[0297] Step 7:

[0298] The terminal displays detailed evaluation results sent from the server to the user. The input is evaluation data from the server, and the output is a feedback screen that the user reviews. This includes accuracy of the response, consideration for customer feelings, and areas for improvement, which the user can review to improve their skills.

[0299] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0300] This invention provides a training system for improving customer service skills in the service industry, using generative artificial intelligence and an emotion engine. The system consists of a server, terminals, and a user, and simulates actual customer interactions while providing feedback that takes into account the user's emotional state.

[0301] The server activates generative artificial intelligence to generate a variety of interaction scenarios. These scenarios include dialogue settings that simulate different customer attributes and situations. These scenarios are presented to the user via a terminal, and the user is configured to select a scenario that suits their training objectives.

[0302] Once the user selects a scenario, a training session begins. During the session, the device utilizes an emotion engine to recognize the user's emotions in real time from their facial expressions and tone of voice. This allows the server to understand the user's emotional state and provide appropriate responses and follow-up. Responses are generated by generative artificial intelligence, providing the user with the most optimal interaction.

[0303] This real-time emotion recognition further improves training efficiency, allowing users to manage emotional stress while enhancing their customer service skills in more realistic situations. Additionally, the emotion data captured by the emotion engine is sent to a server and used to evaluate the user's response tendencies and emotional response capabilities.

[0304] Once training is complete, the server comprehensively evaluates the user's interactions and provides feedback based on the results. This feedback includes specific advice on emotional handling and the quality of interactions. Furthermore, in addition to training history and evaluation information, the server records emotional data in a database, providing information that helps track the user's learning progress and contributes to long-term skill improvement.

[0305] As a specific example, when conducting effective claim handling training, the generative artificial intelligence sets up a customer role with intense emotional changes, and the user attempts to respond to this situation. The emotion engine analyzes the subtle emotional changes shown by the user in real time, and appropriate advice based on this information is provided during and after the training. As a result, the user can not only improve their technical skills but also enhance their coping ability in terms of emotions. In this way, the system greatly contributes to the improvement of the quality of customer service.

[0306] The following explains the processing flow.

[0307] Step 1:

[0308] The server activates the generative artificial intelligence and designs and generates various customer response scenarios. These scenarios incorporate different customer attributes and create a list for the user to select.

[0309] Step 2:

[0310] The terminal displays the scenario list sent from the server to the user. The user selects a scenario that suits their training needs and sends that selection from the terminal to the server.

[0311] Step 3:

[0312] Based on the selected scenario, the server configures an AI character and an interaction environment and prepares to start a training session.

[0313] Step 4:

[0314] When the training session starts, the terminal activates the emotion engine, analyzes the user's expressions and voices in real time, and obtains emotion data. This data is immediately sent to the server.

[0315] Step 5:

[0316] The user interacts with an AI-generated customer through their device. Based on emotional data provided by the emotion engine, the server generates the optimal response each time and adjusts the feedback provided to the user.

[0317] Step 6:

[0318] The server links user responses with emotional data, generates an AI response at the appropriate time, and sends it to the device for display to the user. This allows the user to continue interacting while receiving continuous feedback.

[0319] Step 7:

[0320] Once the training session is complete, the server comprehensively evaluates the user's interaction and emotional data to generate personalized feedback.

[0321] Step 8:

[0322] Feedback is provided to users through their devices and includes suggestions for improving customer service and advice on emotional responses.

[0323] Step 9:

[0324] The server records training history, evaluation information, and sentiment data in a database. Users can view this information on their devices and use it to improve their skills in the future.

[0325] (Example 2)

[0326] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0327] While customer service skills are becoming increasingly important in the service industry, traditional training methods struggle to improve these skills in realistic situations. In particular, there is a growing need to enhance the ability to respond to users' emotional states, but effective methods for achieving this are lacking. To address these issues, there is a need for the development of interactive training systems that analyze users' emotional states and incorporate the results.

[0328] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0329] In this invention, the server includes means for generating multiple dialogue scenarios using generative artificial intelligence, means for analyzing response data received from the user and creating an appropriate response using generative artificial intelligence, and means for analyzing the user's emotional state in real time using an emotion analysis engine and generating feedback based on that data. This makes it possible to provide a realistic simulation environment that takes the user's emotional state into account and to efficiently improve customer service skills.

[0330] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to automatically generate text, such as response scenarios, based on given prompts.

[0331] A "dialogue scenario" is a practice scenario that includes hypothetical conversation content and is used to simulate different customer attributes and situations.

[0332] "Response data" refers to the collection of reactions and input information that a user provides during an interaction.

[0333] An "emotion analysis engine" is a technology that evaluates and detects a user's emotional state in real time from data such as facial expressions and voice tone.

[0334] "Feedback" refers to information that includes evaluations and suggestions for improvement regarding the user's interactions, and is provided as part of training.

[0335] An "interface" is a means or method that provides a way for a user and a system to exchange information.

[0336] This invention is a training system for improving customer service skills in the service industry. The following describes a specific embodiment for carrying out the invention.

[0337] The server runs a generative artificial intelligence model on industry-standard computing equipment. This model utilizes generative AI models such as OpenAI's GPT series to generate customer service dialogue scenarios. An example prompt statement could be, "Generate dialogue scenarios that consider different customer attributes and situations." These generated scenarios will include a variety of customer attributes and response patterns.

[0338] The terminal is a device that functions as a user interface, providing a list of scenarios for the user to select. The terminal has a touchscreen display and audio input / output devices to accept user input. Based on the selected scenario, the terminal activates an emotion analysis engine and analyzes the user's emotional state in real time. This analysis is performed using EmotionAPI and other emotion recognition software. Based on data obtained from the user's facial expressions and tone of voice, the emotional state is quantified and sent to the server.

[0339] The server uses a generative AI model to generate optimal responses and feedback based on received sentiment data and user response data. This feedback includes specific advice on areas for improvement and skills to strengthen. This feedback, delivered through the user interface, allows users to improve their skills step-by-step and practically.

[0340] As a concrete example, during complaint handling training, generative artificial intelligence sets up a simulation role using the prompt, "Generate a conversation scenario that guides an angry customer to calm down." As the user attempts to handle the situation, the device recognizes emotions and continuously provides appropriate advice. This system allows users to efficiently improve their practical skills in customer service.

[0341] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0342] Step 1: Scenario Generation

[0343] The server activates a generative AI model and, using the configured prompt text as input, generates a variety of interaction scenarios. These scenarios include different customer attributes and situations. The output is a virtual dialogue scenario, which is saved in text format.

[0344] Step 2: Presenting the Scenario

[0345] The terminal receives scenarios output from the server and presents them to the user through a user interface. Specifically, it displays a list of scenarios on the terminal's display and allows selection via touch operation. The input is a list of scenarios, and the output is the selected scenario.

[0346] Step 3: Analysis of emotional data

[0347] The device activates an emotion analysis engine based on a scenario selected by the user. It analyzes the user's facial expressions and voice tone in real time and acquires this information as digital data. The input is the user's voice and video, and the output is quantified emotion data.

[0348] Step 4: Generating response feedback

[0349] The server receives emotion data and user responses from the terminal as input and uses a generative AI model to generate optimal feedback. The generated feedback includes areas for improvement and hints for better responses. The output is feedback in text format.

[0350] Step 5: Provide feedback

[0351] The terminal provides the user with feedback obtained from the server. Specifically, it can display advice on the terminal screen and also provide voice guidance. The input is text data of the feedback, and the output is the presentation of visual or auditory information to the user.

[0352] In this way, the system provides users with training in a realistic environment and helps improve their customer service skills based on emotion recognition.

[0353] (Application Example 2)

[0354] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0355] In the service industry, a challenge is providing real-time feedback to efficiently improve employees' customer service skills. Existing training systems lack the ability to handle diverse customer attributes and improve emotional care skills, and there is a lack of appropriate feedback that utilizes emotional information.

[0356] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0357] In this invention, the server includes means for generating multiple response scenarios using generative artificial intelligence, means for recognizing the user's emotional state in real time and optimizing the interaction using that information, and means for providing real-time feedback through an adaptive human interface. This enables users to efficiently improve their customer service skills.

[0358] "Generative artificial intelligence" is an artificial intelligence technology that generates responses based on user interaction and provides various response scenarios.

[0359] A "customer service scenario" is a hypothetical response case designed to simulate actual customer service situations and improve user skills.

[0360] "Emotional state" refers to the psychological and emotional condition that can be determined from the user's facial expressions and voice.

[0361] "Interaction" refers to the dynamic, two-way exchange that takes place between the user and the system.

[0362] An "adaptive human interface" is a point of contact between humans and machines that dynamically changes according to the user's situation to provide optimal feedback.

[0363] "Feedback" is a means of providing information to improve user performance based on the results of training sessions.

[0364] The system for realizing this invention primarily consists of a server, a terminal, and a user. This system is designed to improve customer service skills in the service industry. The server uses generative artificial intelligence to generate diverse response scenarios and provides them to the user. The user can access these scenarios through the terminal and implement appropriate responses.

[0365] The device uses an emotion engine to analyze the user's voice tone and facial expressions in real time. This emotion engine accurately captures the user's emotional state and transmits it to the server. This allows the server to generate responses and feedback tailored to the user's emotional state, providing a more realistic training environment. The server also maintains the user's training history and evaluation information to support long-term skill improvement.

[0366] As a concrete example, consider a scenario where a cafe employee practices handling customer complaints due to order errors. The server generates a scenario titled "Generating AI Model, Prompt Sentence," featuring a customer with rapidly changing emotions. The user selects this scenario and responds through interaction with the virtual customer. During this process, the terminal detects the user's emotional changes and provides appropriate feedback. For example, a prompt sentence such as "Please tell me how to respond when a customer is served the wrong drink" could be used. In this way, the system of the present invention enhances the user's ability to respond to emotional issues and helps them acquire better customer service skills.

[0367] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0368] Step 1:

[0369] The server generates diverse interaction scenarios using generative artificial intelligence. The server receives user training objectives and configuration parameters as input. Based on these inputs, the generative AI model generates scenarios with appropriate prompts and different customer attributes. The output consists of multiple virtual scenarios that the user can select from.

[0370] Step 2:

[0371] The user uses a terminal to select a response scenario provided by the server. During this selection process, the user interface displays an overview of each scenario. The terminal receives the user's selection information, and the output is a message indicating that the selected scenario is ready to begin.

[0372] Step 3:

[0373] The device uses an emotion engine to analyze the user's facial expressions and voice in real time. The input to the device consists of the user's raw voice data and facial image data. This data is processed using an emotion analysis algorithm to identify the user's emotional state. The output is the analyzed emotion data.

[0374] Step 4:

[0375] The server receives the analyzed emotion data and generates optimal feedback tailored to the user's emotional state. The server receives the result of the emotion analysis as input. Based on this data, the server utilizes a generation AI model to generate appropriate responses and follow-ups for the user. The output is a feedback message.

[0376] Step 5:

[0377] Feedback is provided to the user in real time via the device. Through this feedback, the user has the opportunity to adjust their response methods. The input to the device is a feedback message sent from the server, and the output is improvement guidelines provided to the user.

[0378] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0379] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0380] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0381] [Third Embodiment]

[0382] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0383] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0384] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0385] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0386] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0387] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0388] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0389] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0390] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0391] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0392] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0393] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0394] This invention is a training system aimed at effectively improving customer service skills in the service industry. The system consists of a server, terminals, and users, and utilizes a generative artificial intelligence model to provide diverse customer service scenarios.

[0395] The server runs generative artificial intelligence to generate various interaction scenarios. These scenarios have different dialogues based on various customer attributes and needs. The server sends these scenarios to the terminal, allowing the user to select one. The user can then select a scenario that suits their skill level and learning objectives via the terminal and begin training.

[0396] During training sessions, users interact with a customer role played by AI through their device. User input data is sent to a server in real time and analyzed by generative artificial intelligence. Based on this analysis, the server generates an appropriate response and sends it to the device. This allows users to experience virtual customer service and develop skills relevant to actual work.

[0397] Once the training is complete, the server evaluates the user's responses and provides specific feedback through the terminal. This feedback includes aspects such as the accuracy of the response, consideration for customer emotions, and areas for improvement, helping the user improve their skills. Furthermore, the server stores training history and evaluation information in a database, allowing users to track their learning progress.

[0398] As a concrete example, when a new staff member is being trained to deal with a "highly dissatisfied customer," the generative artificial intelligence sets up a virtual customer with specific reasons for dissatisfaction and provides the user with a role-playing scenario. The user attempts to respond from their terminal, and the server evaluates their response. Through this process, the user can directly and quickly improve their problem-solving abilities. In this way, the system enables efficient skill improvement and contributes to improving the quality of customer service.

[0399] The following describes the processing flow.

[0400] Step 1:

[0401] The server activates a generative artificial intelligence model to generate various customer interaction scenarios. This includes setting up dialogues based on different customer attributes and their respective situations.

[0402] Step 2:

[0403] The server sends the generated scenarios to the terminal as a list. The terminal displays this list to the user, allowing the user to select the scenario they want to train.

[0404] Step 3:

[0405] The user selects a scenario through their device. The selection information is sent from the device to the server, which then configures the AI ​​character based on the selected scenario.

[0406] Step 4:

[0407] The training session begins, and the user interacts with a customer role played by AI via the device. This interaction is conducted via voice or text input.

[0408] Step 5:

[0409] User input data is sent from the terminal to the server in real time. The server analyzes the input and uses an AI model to generate appropriate responses as a customer.

[0410] Step 6:

[0411] The server sends the generated response to the terminal. The terminal then displays or audibly presents this response to the user, allowing the conversation to continue.

[0412] Step 7:

[0413] After the session ends, the server evaluates the user's response. This evaluation includes metrics such as accuracy, level of consideration, and room for improvement.

[0414] Step 8:

[0415] Based on the evaluation results, the server generates feedback for the user. This feedback is provided to the user via the terminal.

[0416] Step 9:

[0417] The server records training history and evaluation information in a database. Users can access this information from their terminals and use it as a guide for self-study.

[0418] (Example 1)

[0419] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0420] In today's service industry, the quality of customer service is a crucial factor directly linked to a company's success. However, traditional training methods have made it difficult to customize training content to the individual needs and skill levels of employees. Furthermore, there is a challenge in developing realistic customer service skills without actual interaction with customers. In addition, there is a lack of mechanisms for employees to monitor their own progress and continuously improve their skills.

[0421] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0422] In this invention, the server includes means for generating dialogue scenarios with different conditions using generative artificial intelligence, means for analyzing data received from an information processing device and generating the optimal dialogue using generative artificial intelligence, and means for conducting training sessions for users and determining the results of those sessions. This enables customized training tailored to individual users, progress tracking, and practical and realistic customer service training.

[0423] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new data and information based on given input. This technology allows for the automatic creation of diverse scenarios and responses.

[0424] "Conditions" refer to the various elements and circumstances considered when forming a particular scenario or response, which allows the resulting content to have different characteristics.

[0425] An "information processing device" refers to equipment that receives data and appropriately analyzes and processes it, and mainly includes hardware such as computers and servers.

[0426] A "dialogue scenario" refers to a series of dialogues designed with a specific purpose in mind, providing users with realistic training through content that reflects different conditions.

[0427] "Progress" refers to the level of improvement in skills and abilities that users have achieved through training, and is evaluated by comparing it to their past state.

[0428] "Judgment" refers to the process by which an information processing device evaluates the results of a training session and analyzes the findings, clearly indicating areas for improvement and the degree of achievement.

[0429] To implement this invention, it is necessary to configure a system using a generative artificial intelligence model. The server is responsible for driving this generative AI model and generating various response scenarios. Specifically, the server takes prompt text as input, and the AI ​​model generates a variety of scenarios necessary for customer service. An example of such prompt text would be, "Generate scenarios for customer service training in the service industry. Please include customer attributes and needs." The generated scenarios reflect different customer attributes and situations, and aim to improve the user's ability to handle various cases.

[0430] The terminal is a device that receives scenarios sent from the server and provides them to the user. A list of selectable scenarios is displayed on the terminal, and the user can choose a scenario according to their learning objectives.

[0431] The user interacts with a virtual customer, played by AI, via their device. The user's responses are sent to the server in real time, where a generative AI model analyzes them and generates the next response. Through this process, the user can experience virtual customer service. Once training is complete, the server evaluates the user's responses and provides specific feedback via the device. This feedback includes aspects such as the accuracy of the response, consideration of customer emotions, and areas for improvement.

[0432] A concrete example is a scenario where new staff members undergo training to handle "highly dissatisfied customers." In this scenario, generative artificial intelligence sets up a virtual customer with specific reasons for dissatisfaction and provides the user with role-playing through dialogue. Through this process, users can improve their problem-solving abilities in a practical and rapid manner. This system is expected to efficiently improve skills in the service industry and directly contribute to improving the quality of work.

[0433] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0434] Step 1:

[0435] The server supplies prompt text as input to the AI ​​model. This input allows the AI ​​model to generate conversational scenarios based on customer attributes and needs. The generated scenarios are saved as server output and ready to be provided to the user later.

[0436] Step 2:

[0437] The server sends the generated scenarios to the terminal. On the terminal, multiple scenarios are displayed in a list format, allowing the user to select one. Through this data transfer, the terminal plays the role of providing the user with choices.

[0438] Step 3:

[0439] The user uses their device to select a scenario that suits their needs and learning goals, and then begins training. The selected scenario is notified from the device to the server based on the user's selection, and the server receives a signal to begin training.

[0440] Step 4:

[0441] Based on scenario information from the server, the terminal presents the user with the first question from a virtual customer played by AI. Once the user's input is received by the terminal, it is sent to the server.

[0442] Step 5:

[0443] The server receives user input and analyzes it using generative artificial intelligence. Based on this analysis, it generates the next appropriate customer response. The generated response is sent to the terminal as server output, and the interaction with the user continues.

[0444] Step 6:

[0445] After training is complete, the server evaluates all interactions. This evaluation includes the appropriateness of the interaction and the level of understanding of customer sentiment. The evaluation results are sent from the server to the terminal, which then presents them to the user as feedback.

[0446] Step 7:

[0447] The server stores data obtained during training sessions in a database. This includes the user's interaction history and evaluation results. The stored data allows users to check their skill progress from their devices. Access to this information facilitates continuous skill improvement.

[0448] (Application Example 1)

[0449] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0450] Improving customer service skills is crucial in customer service roles, but there is a challenge in conducting realistic training that closely reflects actual work situations. Furthermore, developing the ability to respond flexibly to different customer attributes requires the creation of diverse scenarios, but efficiently implementing these is difficult.

[0451] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0452] In this invention, the server includes means for generating multiple response scenarios using generative artificial intelligence, means for providing virtual dialogues to the user in real time via a visual interface device, and means for providing real-time feedback via the visual interface device. This enables realistic training in an actual work environment and rapid skill improvement of staff.

[0453] "Generative artificial intelligence" is a technology that automatically generates diverse dialogue scenarios and creates appropriate responses through interaction with the user.

[0454] "Response data" refers to information received from the user, which forms the basis for generative artificial intelligence to analyze and generate appropriate dialogue.

[0455] A "training session" is a process in which a user interacts with generative artificial intelligence to improve their skills.

[0456] "Evaluation results" refer to data used to assess a user's communication skills during a training session and to provide feedback on those skills.

[0457] A "visual interface device" is a device that allows users to visually experience virtual interactions, and in most cases refers to devices such as smart glasses.

[0458] "Real-time feedback" is feedback generated instantly in response to user actions and responses, providing immediate points for improvement to enhance skills.

[0459] The system for implementing this invention mainly consists of a server, a visual interface device (e.g., smart glasses), and a user terminal. The server is responsible for generating multiple interaction scenarios using a generative AI model. This AI model automatically generates diverse customer scenarios, creating an environment in which users can interact with virtual customers with different attributes.

[0460] The visual interface device is designed to provide users with virtual interactions in real time, allowing them to communicate with the system. The user's spoken content and responses are transmitted to a server via the terminal, which analyzes them using a generative AI model. The server then generates appropriate responses and returns them in real time, providing a realistic customer service training environment.

[0461] Feedback is provided immediately through a visual interface device, allowing users to make corrections and improvements in real time. After the training session ends, the server evaluates the user's responses and sends feedback based on the evaluation results to the terminal. This feedback includes accuracy of responses and areas for improvement.

[0462] One concrete example is a scenario in which department store sales staff use smart glasses to interact with virtual customers with different personalities and needs. During this training, users can constantly try out appropriate responses and hone their skills. An example of a prompt might be, "The customer is having trouble with the fit of their jeans. How would the customer express their dissatisfaction?"

[0463] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0464] Step 1:

[0465] The server generates multiple interaction scenarios using generative artificial intelligence. It receives diverse customer attribute and needs information as input and analyzes it using a generative AI model. As output, it generates unique dialogue scenarios, which are then prepared for the user to select in the next step.

[0466] Step 2:

[0467] The terminal presents the user with response scenarios received from the server. The input is scenario data from the server, and the output is a scenario selection screen on the user interface. The user selects a scenario appropriate to their level through the terminal and begins training.

[0468] Step 3:

[0469] The user interacts with a virtual customer using a visual interface device. Input consists of audio and text data of the user's responses, which the visual interface transmits to the server in real time. Output is recorded as the user's response and sent to the next analysis step.

[0470] Step 4:

[0471] The server analyzes response data received from the user. The input is real-time text or audio data. Data analysis is performed, and an optimized response is generated using generative artificial intelligence. Prompts are used to improve the dialogue generation. The output is returned to the user as the next message from a virtual customer.

[0472] Step 5:

[0473] The visual interface device provides the user with generated responses in real time and allows them to evaluate the responses. The input is the response from the server, and the output is feedback information that the user sees. Furthermore, suggested areas for improvement are presented, allowing the user to improve the response on the spot.

[0474] Step 6:

[0475] Once the training session is complete, the server comprehensively evaluates the overall interaction. The input consists of all responses and interaction data recorded during training. Based on the evaluation criteria, the server performs analysis and sends detailed evaluation results and feedback to the terminal as output.

[0476] Step 7:

[0477] The terminal displays detailed evaluation results sent from the server to the user. The input is evaluation data from the server, and the output is a feedback screen that the user reviews. This includes accuracy of the response, consideration for customer feelings, and areas for improvement, which the user can review to improve their skills.

[0478] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0479] This invention provides a training system for improving customer service skills in the service industry, using generative artificial intelligence and an emotion engine. The system consists of a server, terminals, and a user, and simulates actual customer interactions while providing feedback that takes into account the user's emotional state.

[0480] The server activates generative artificial intelligence to generate a variety of interaction scenarios. These scenarios include dialogue settings that simulate different customer attributes and situations. These scenarios are presented to the user via a terminal, and the user is configured to select a scenario that suits their training objectives.

[0481] Once the user selects a scenario, a training session begins. During the session, the device utilizes an emotion engine to recognize the user's emotions in real time from their facial expressions and tone of voice. This allows the server to understand the user's emotional state and provide appropriate responses and follow-up. Responses are generated by generative artificial intelligence, providing the user with the most optimal interaction.

[0482] This real-time emotion recognition further improves training efficiency, allowing users to manage emotional stress while enhancing their customer service skills in more realistic situations. Additionally, the emotion data captured by the emotion engine is sent to a server and used to evaluate the user's response tendencies and emotional response capabilities.

[0483] Once training is complete, the server comprehensively evaluates the user's interactions and provides feedback based on the results. This feedback includes specific advice on emotional handling and the quality of interactions. Furthermore, in addition to training history and evaluation information, the server records emotional data in a database, providing information that helps track the user's learning progress and contributes to long-term skill improvement.

[0484] As a concrete example, in training on effective complaint handling, the generative artificial intelligence sets up a customer role with volatile emotions, and the user attempts to respond to this situation. The emotion engine analyzes the subtle emotional changes the user exhibits in real time, and appropriate advice based on this information is provided during and after the training. This allows the user to improve not only their technical skills but also their emotional response capabilities. In this way, the system makes a significant contribution to improving the quality of service in the hospitality industry.

[0485] The following describes the processing flow.

[0486] Step 1:

[0487] The server activates generative artificial intelligence to design and generate various customer interaction scenarios. These scenarios incorporate different customer attributes and create a list for the user to select from.

[0488] Step 2:

[0489] The terminal displays a list of scenarios sent from the server to the user. The user selects a scenario that suits their training needs and sends that selection from the terminal to the server.

[0490] Step 3:

[0491] The server configures the AI ​​character and interaction environment based on the selected scenario, and prepares to start the training session.

[0492] Step 4:

[0493] When a training session begins, the device activates its emotion engine, analyzing the user's facial expressions and voice in real time to acquire emotion data. This data is immediately sent to the server.

[0494] Step 5:

[0495] The user interacts with an AI-generated customer through their device. Based on emotional data provided by the emotion engine, the server generates the optimal response each time and adjusts the feedback provided to the user.

[0496] Step 6:

[0497] The server links user responses with emotional data, generates an AI response at the appropriate time, and sends it to the device for display to the user. This allows the user to continue interacting while receiving continuous feedback.

[0498] Step 7:

[0499] Once the training session is complete, the server comprehensively evaluates the user's interaction and emotional data to generate personalized feedback.

[0500] Step 8:

[0501] Feedback is provided to users through their devices and includes suggestions for improving customer service and advice on emotional responses.

[0502] Step 9:

[0503] The server records training history, evaluation information, and sentiment data in a database. Users can view this information on their devices and use it to improve their skills in the future.

[0504] (Example 2)

[0505] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0506] While customer service skills are becoming increasingly important in the service industry, traditional training methods struggle to improve these skills in realistic situations. In particular, there is a growing need to enhance the ability to respond to users' emotional states, but effective methods for achieving this are lacking. To address these issues, there is a need for the development of interactive training systems that analyze users' emotional states and incorporate the results.

[0507] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0508] In this invention, the server includes means for generating multiple dialogue scenarios using generative artificial intelligence, means for analyzing response data received from the user and creating an appropriate response using generative artificial intelligence, and means for analyzing the user's emotional state in real time using an emotion analysis engine and generating feedback based on that data. This makes it possible to provide a realistic simulation environment that takes the user's emotional state into account and to efficiently improve customer service skills.

[0509] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to automatically generate text, such as response scenarios, based on given prompts.

[0510] A "dialogue scenario" is a practice scenario that includes hypothetical conversation content and is used to simulate different customer attributes and situations.

[0511] "Response data" refers to the collection of reactions and input information that a user provides during an interaction.

[0512] An "emotion analysis engine" is a technology that evaluates and detects a user's emotional state in real time from data such as facial expressions and voice tone.

[0513] "Feedback" refers to information that includes evaluations and suggestions for improvement regarding the user's interactions, and is provided as part of training.

[0514] An "interface" is a means or method that provides a way for a user and a system to exchange information.

[0515] This invention is a training system for improving customer service skills in the service industry. The following describes a specific embodiment for carrying out the invention.

[0516] The server runs a generative artificial intelligence model on industry-standard computing equipment. This model utilizes generative AI models such as OpenAI's GPT series to generate customer service dialogue scenarios. An example prompt statement could be, "Generate dialogue scenarios that consider different customer attributes and situations." These generated scenarios will include a variety of customer attributes and response patterns.

[0517] The terminal is a device that functions as a user interface, providing a list of scenarios for the user to select. The terminal has a touchscreen display and audio input / output devices to accept user input. Based on the selected scenario, the terminal activates an emotion analysis engine and analyzes the user's emotional state in real time. This analysis is performed using EmotionAPI and other emotion recognition software. Based on data obtained from the user's facial expressions and tone of voice, the emotional state is quantified and sent to the server.

[0518] The server uses a generative AI model to generate optimal responses and feedback based on received sentiment data and user response data. This feedback includes specific advice on areas for improvement and skills to strengthen. This feedback, delivered through the user interface, allows users to improve their skills step-by-step and practically.

[0519] As a concrete example, during complaint handling training, generative artificial intelligence sets up a simulation role using the prompt, "Generate a conversation scenario that guides an angry customer to calm down." As the user attempts to handle the situation, the device recognizes emotions and continuously provides appropriate advice. This system allows users to efficiently improve their practical skills in customer service.

[0520] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0521] Step 1: Scenario Generation

[0522] The server activates a generative AI model and, using the configured prompt text as input, generates a variety of interaction scenarios. These scenarios include different customer attributes and situations. The output is a virtual dialogue scenario, which is saved in text format.

[0523] Step 2: Presenting the Scenario

[0524] The terminal receives scenarios output from the server and presents them to the user through a user interface. Specifically, it displays a list of scenarios on the terminal's display and allows selection via touch operation. The input is a list of scenarios, and the output is the selected scenario.

[0525] Step 3: Analysis of emotional data

[0526] The device activates an emotion analysis engine based on a scenario selected by the user. It analyzes the user's facial expressions and voice tone in real time and acquires this information as digital data. The input is the user's voice and video, and the output is quantified emotion data.

[0527] Step 4: Generating response feedback

[0528] The server receives emotion data and user responses from the terminal as input and uses a generative AI model to generate optimal feedback. The generated feedback includes areas for improvement and hints for better responses. The output is feedback in text format.

[0529] Step 5: Provide feedback

[0530] The terminal provides the user with feedback obtained from the server. Specifically, it can display advice on the terminal screen and also provide voice guidance. The input is text data of the feedback, and the output is the presentation of visual or auditory information to the user.

[0531] In this way, the system provides users with training in a realistic environment and helps improve their customer service skills based on emotion recognition.

[0532] (Application Example 2)

[0533] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0534] In the service industry, a challenge is providing real-time feedback to efficiently improve employees' customer service skills. Existing training systems lack the ability to handle diverse customer attributes and improve emotional care skills, and there is a lack of appropriate feedback that utilizes emotional information.

[0535] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0536] In this invention, the server includes means for generating multiple response scenarios using generative artificial intelligence, means for recognizing the user's emotional state in real time and optimizing the interaction using that information, and means for providing real-time feedback through an adaptive human interface. This enables users to efficiently improve their customer service skills.

[0537] "Generative artificial intelligence" is an artificial intelligence technology that generates responses based on user interaction and provides various response scenarios.

[0538] A "customer service scenario" is a hypothetical response case designed to simulate actual customer service situations and improve user skills.

[0539] "Emotional state" refers to the psychological and emotional condition that can be determined from the user's facial expressions and voice.

[0540] "Interaction" refers to the dynamic, two-way exchange that takes place between the user and the system.

[0541] An "adaptive human interface" is a point of contact between humans and machines that dynamically changes according to the user's situation to provide optimal feedback.

[0542] "Feedback" is a means of providing information to improve user performance based on the results of training sessions.

[0543] The system for realizing this invention primarily consists of a server, a terminal, and a user. This system is designed to improve customer service skills in the service industry. The server uses generative artificial intelligence to generate diverse response scenarios and provides them to the user. The user can access these scenarios through the terminal and implement appropriate responses.

[0544] The device uses an emotion engine to analyze the user's voice tone and facial expressions in real time. This emotion engine accurately captures the user's emotional state and transmits it to the server. This allows the server to generate responses and feedback tailored to the user's emotional state, providing a more realistic training environment. The server also maintains the user's training history and evaluation information to support long-term skill improvement.

[0545] As a concrete example, consider a scenario where a cafe employee practices handling customer complaints due to order errors. The server generates a scenario titled "Generating AI Model, Prompt Sentence," featuring a customer with rapidly changing emotions. The user selects this scenario and responds through interaction with the virtual customer. During this process, the terminal detects the user's emotional changes and provides appropriate feedback. For example, a prompt sentence such as "Please tell me how to respond when a customer is served the wrong drink" could be used. In this way, the system of the present invention enhances the user's ability to respond to emotional issues and helps them acquire better customer service skills.

[0546] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0547] Step 1:

[0548] The server generates diverse interaction scenarios using generative artificial intelligence. The server receives user training objectives and configuration parameters as input. Based on these inputs, the generative AI model generates scenarios with appropriate prompts and different customer attributes. The output consists of multiple virtual scenarios that the user can select from.

[0549] Step 2:

[0550] The user uses a terminal to select a response scenario provided by the server. During this selection process, the user interface displays an overview of each scenario. The terminal receives the user's selection information, and the output is a message indicating that the selected scenario is ready to begin.

[0551] Step 3:

[0552] The device uses an emotion engine to analyze the user's facial expressions and voice in real time. The input to the device consists of the user's raw voice data and facial image data. This data is processed using an emotion analysis algorithm to identify the user's emotional state. The output is the analyzed emotion data.

[0553] Step 4:

[0554] The server receives the analyzed emotion data and generates optimal feedback tailored to the user's emotional state. The server receives the result of the emotion analysis as input. Based on this data, the server utilizes a generation AI model to generate appropriate responses and follow-ups for the user. The output is a feedback message.

[0555] Step 5:

[0556] Feedback is provided to the user in real time via the device. Through this feedback, the user has the opportunity to adjust their response methods. The input to the device is a feedback message sent from the server, and the output is improvement guidelines provided to the user.

[0557] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0558] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0559] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0560] [Fourth Embodiment]

[0561] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0562] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0563] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0564] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0565] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0566] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0567] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0568] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0569] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0570] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0571] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0572] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0573] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0574] This invention is a training system aimed at effectively improving customer service skills in the service industry. The system consists of a server, terminals, and users, and utilizes a generative artificial intelligence model to provide diverse customer service scenarios.

[0575] The server runs generative artificial intelligence to generate various interaction scenarios. These scenarios have different dialogues based on various customer attributes and needs. The server sends these scenarios to the terminal, allowing the user to select one. The user can then select a scenario that suits their skill level and learning objectives via the terminal and begin training.

[0576] During training sessions, users interact with a customer role played by AI through their device. User input data is sent to a server in real time and analyzed by generative artificial intelligence. Based on this analysis, the server generates an appropriate response and sends it to the device. This allows users to experience virtual customer service and develop skills relevant to actual work.

[0577] Once the training is complete, the server evaluates the user's responses and provides specific feedback through the terminal. This feedback includes aspects such as the accuracy of the response, consideration for customer emotions, and areas for improvement, helping the user improve their skills. Furthermore, the server stores training history and evaluation information in a database, allowing users to track their learning progress.

[0578] As a concrete example, when a new staff member is being trained to deal with a "highly dissatisfied customer," the generative artificial intelligence sets up a virtual customer with specific reasons for dissatisfaction and provides the user with a role-playing scenario. The user attempts to respond from their terminal, and the server evaluates their response. Through this process, the user can directly and quickly improve their problem-solving abilities. In this way, the system enables efficient skill improvement and contributes to improving the quality of customer service.

[0579] The following describes the processing flow.

[0580] Step 1:

[0581] The server activates a generative artificial intelligence model to generate various customer interaction scenarios. This includes setting up dialogues based on different customer attributes and their respective situations.

[0582] Step 2:

[0583] The server sends the generated scenarios to the terminal as a list. The terminal displays this list to the user, allowing the user to select the scenario they want to train.

[0584] Step 3:

[0585] The user selects a scenario through their device. The selection information is sent from the device to the server, which then configures the AI ​​character based on the selected scenario.

[0586] Step 4:

[0587] The training session begins, and the user interacts with a customer role played by AI via the device. This interaction is conducted via voice or text input.

[0588] Step 5:

[0589] User input data is sent from the terminal to the server in real time. The server analyzes the input and uses an AI model to generate appropriate responses as a customer.

[0590] Step 6:

[0591] The server sends the generated response to the terminal. The terminal then displays or audibly presents this response to the user, allowing the conversation to continue.

[0592] Step 7:

[0593] After the session ends, the server evaluates the user's response. This evaluation includes metrics such as accuracy, level of consideration, and room for improvement.

[0594] Step 8:

[0595] Based on the evaluation results, the server generates feedback for the user. This feedback is provided to the user via the terminal.

[0596] Step 9:

[0597] The server records training history and evaluation information in a database. Users can access this information from their terminals and use it as a guide for self-study.

[0598] (Example 1)

[0599] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0600] In today's service industry, the quality of customer service is a crucial factor directly linked to a company's success. However, traditional training methods have made it difficult to customize training content to the individual needs and skill levels of employees. Furthermore, there is a challenge in developing realistic customer service skills without actual interaction with customers. In addition, there is a lack of mechanisms for employees to monitor their own progress and continuously improve their skills.

[0601] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0602] In this invention, the server includes means for generating dialogue scenarios with different conditions using generative artificial intelligence, means for analyzing data received from an information processing device and generating the optimal dialogue using generative artificial intelligence, and means for conducting training sessions for users and determining the results of those sessions. This enables customized training tailored to individual users, progress tracking, and practical and realistic customer service training.

[0603] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new data and information based on given input. This technology allows for the automatic creation of diverse scenarios and responses.

[0604] "Conditions" refer to the various elements and circumstances considered when forming a particular scenario or response, which allows the resulting content to have different characteristics.

[0605] An "information processing device" refers to equipment that receives data and appropriately analyzes and processes it, and mainly includes hardware such as computers and servers.

[0606] A "dialogue scenario" refers to a series of dialogues designed with a specific purpose in mind, providing users with realistic training through content that reflects different conditions.

[0607] "Progress" refers to the level of improvement in skills and abilities that users have achieved through training, and is evaluated by comparing it to their past state.

[0608] "Judgment" refers to the process by which an information processing device evaluates the results of a training session and analyzes the findings, clearly indicating areas for improvement and the degree of achievement.

[0609] To implement this invention, it is necessary to configure a system using a generative artificial intelligence model. The server is responsible for driving this generative AI model and generating various response scenarios. Specifically, the server takes prompt text as input, and the AI ​​model generates a variety of scenarios necessary for customer service. An example of such prompt text would be, "Generate scenarios for customer service training in the service industry. Please include customer attributes and needs." The generated scenarios reflect different customer attributes and situations, and aim to improve the user's ability to handle various cases.

[0610] The terminal is a device that receives scenarios sent from the server and provides them to the user. A list of selectable scenarios is displayed on the terminal, and the user can choose a scenario according to their learning objectives.

[0611] The user interacts with a virtual customer, played by AI, via their device. The user's responses are sent to the server in real time, where a generative AI model analyzes them and generates the next response. Through this process, the user can experience virtual customer service. Once training is complete, the server evaluates the user's responses and provides specific feedback via the device. This feedback includes aspects such as the accuracy of the response, consideration of customer emotions, and areas for improvement.

[0612] A concrete example is a scenario where new staff members undergo training to handle "highly dissatisfied customers." In this scenario, generative artificial intelligence sets up a virtual customer with specific reasons for dissatisfaction and provides the user with role-playing through dialogue. Through this process, users can improve their problem-solving abilities in a practical and rapid manner. This system is expected to efficiently improve skills in the service industry and directly contribute to improving the quality of work.

[0613] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0614] Step 1:

[0615] The server supplies prompt text as input to the AI ​​model. This input allows the AI ​​model to generate conversational scenarios based on customer attributes and needs. The generated scenarios are saved as server output and ready to be provided to the user later.

[0616] Step 2:

[0617] The server sends the generated scenarios to the terminal. On the terminal, multiple scenarios are displayed in a list format, allowing the user to select one. Through this data transfer, the terminal plays the role of providing the user with choices.

[0618] Step 3:

[0619] The user uses their device to select a scenario that suits their needs and learning goals, and then begins training. The selected scenario is notified from the device to the server based on the user's selection, and the server receives a signal to begin training.

[0620] Step 4:

[0621] Based on scenario information from the server, the terminal presents the user with the first question from a virtual customer played by AI. Once the user's input is received by the terminal, it is sent to the server.

[0622] Step 5:

[0623] The server receives user input and analyzes it using generative artificial intelligence. Based on this analysis, it generates the next appropriate customer response. The generated response is sent to the terminal as server output, and the interaction with the user continues.

[0624] Step 6:

[0625] After training is complete, the server evaluates all interactions. This evaluation includes the appropriateness of the interaction and the level of understanding of customer sentiment. The evaluation results are sent from the server to the terminal, which then presents them to the user as feedback.

[0626] Step 7:

[0627] The server stores data obtained during training sessions in a database. This includes the user's interaction history and evaluation results. The stored data allows users to check their skill progress from their devices. Access to this information facilitates continuous skill improvement.

[0628] (Application Example 1)

[0629] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0630] Improving customer service skills is crucial in customer service roles, but there is a challenge in conducting realistic training that closely reflects actual work situations. Furthermore, developing the ability to respond flexibly to different customer attributes requires the creation of diverse scenarios, but efficiently implementing these is difficult.

[0631] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0632] In this invention, the server includes means for generating multiple response scenarios using generative artificial intelligence, means for providing virtual dialogues to the user in real time via a visual interface device, and means for providing real-time feedback via the visual interface device. This enables realistic training in an actual work environment and rapid skill improvement of staff.

[0633] "Generative artificial intelligence" is a technology that automatically generates diverse dialogue scenarios and creates appropriate responses through interaction with the user.

[0634] "Response data" refers to information received from the user, which forms the basis for generative artificial intelligence to analyze and generate appropriate dialogue.

[0635] A "training session" is a process in which a user interacts with generative artificial intelligence to improve their skills.

[0636] "Evaluation results" refer to data used to assess a user's communication skills during a training session and to provide feedback on those skills.

[0637] A "visual interface device" is a device that allows users to visually experience virtual interactions, and in most cases refers to devices such as smart glasses.

[0638] "Real-time feedback" is feedback generated instantly in response to user actions and responses, providing immediate points for improvement to enhance skills.

[0639] The system for implementing this invention mainly consists of a server, a visual interface device (e.g., smart glasses), and a user terminal. The server is responsible for generating multiple interaction scenarios using a generative AI model. This AI model automatically generates diverse customer scenarios, creating an environment in which users can interact with virtual customers with different attributes.

[0640] The visual interface device is designed to provide users with virtual interactions in real time, allowing them to communicate with the system. The user's spoken content and responses are transmitted to a server via the terminal, which analyzes them using a generative AI model. The server then generates appropriate responses and returns them in real time, providing a realistic customer service training environment.

[0641] Feedback is provided immediately through a visual interface device, allowing users to make corrections and improvements in real time. After the training session ends, the server evaluates the user's responses and sends feedback based on the evaluation results to the terminal. This feedback includes accuracy of responses and areas for improvement.

[0642] One concrete example is a scenario in which department store sales staff use smart glasses to interact with virtual customers with different personalities and needs. During this training, users can constantly try out appropriate responses and hone their skills. An example of a prompt might be, "The customer is having trouble with the fit of their jeans. How would the customer express their dissatisfaction?"

[0643] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0644] Step 1:

[0645] The server generates multiple interaction scenarios using generative artificial intelligence. It receives diverse customer attribute and needs information as input and analyzes it using a generative AI model. As output, it generates unique dialogue scenarios, which are then prepared for the user to select in the next step.

[0646] Step 2:

[0647] The terminal presents the user with response scenarios received from the server. The input is scenario data from the server, and the output is a scenario selection screen on the user interface. The user selects a scenario appropriate to their level through the terminal and begins training.

[0648] Step 3:

[0649] The user interacts with a virtual customer using a visual interface device. Input consists of audio and text data of the user's responses, which the visual interface transmits to the server in real time. Output is recorded as the user's response and sent to the next analysis step.

[0650] Step 4:

[0651] The server analyzes response data received from the user. The input is real-time text or audio data. Data analysis is performed, and an optimized response is generated using generative artificial intelligence. Prompts are used to improve the dialogue generation. The output is returned to the user as the next message from a virtual customer.

[0652] Step 5:

[0653] The visual interface device provides the user with generated responses in real time and allows them to evaluate the responses. The input is the response from the server, and the output is feedback information that the user sees. Furthermore, suggested areas for improvement are presented, allowing the user to improve the response on the spot.

[0654] Step 6:

[0655] Once the training session is complete, the server comprehensively evaluates the overall interaction. The input consists of all responses and interaction data recorded during training. Based on the evaluation criteria, the server performs analysis and sends detailed evaluation results and feedback to the terminal as output.

[0656] Step 7:

[0657] The terminal displays detailed evaluation results sent from the server to the user. The input is evaluation data from the server, and the output is a feedback screen that the user reviews. This includes accuracy of the response, consideration for customer feelings, and areas for improvement, which the user can review to improve their skills.

[0658] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0659] This invention provides a training system for improving customer service skills in the service industry, using generative artificial intelligence and an emotion engine. The system consists of a server, terminals, and a user, and simulates actual customer interactions while providing feedback that takes into account the user's emotional state.

[0660] The server activates generative artificial intelligence to generate a variety of interaction scenarios. These scenarios include dialogue settings that simulate different customer attributes and situations. These scenarios are presented to the user via a terminal, and the user is configured to select a scenario that suits their training objectives.

[0661] Once the user selects a scenario, a training session begins. During the session, the device utilizes an emotion engine to recognize the user's emotions in real time from their facial expressions and tone of voice. This allows the server to understand the user's emotional state and provide appropriate responses and follow-up. Responses are generated by generative artificial intelligence, providing the user with the most optimal interaction.

[0662] This real-time emotion recognition further improves training efficiency, allowing users to manage emotional stress while enhancing their customer service skills in more realistic situations. Additionally, the emotion data captured by the emotion engine is sent to a server and used to evaluate the user's response tendencies and emotional response capabilities.

[0663] Once training is complete, the server comprehensively evaluates the user's interactions and provides feedback based on the results. This feedback includes specific advice on emotional handling and the quality of interactions. Furthermore, in addition to training history and evaluation information, the server records emotional data in a database, providing information that helps track the user's learning progress and contributes to long-term skill improvement.

[0664] As a concrete example, in training on effective complaint handling, the generative artificial intelligence sets up a customer role with volatile emotions, and the user attempts to respond to this situation. The emotion engine analyzes the subtle emotional changes the user exhibits in real time, and appropriate advice based on this information is provided during and after the training. This allows the user to improve not only their technical skills but also their emotional response capabilities. In this way, the system makes a significant contribution to improving the quality of service in the hospitality industry.

[0665] The following describes the processing flow.

[0666] Step 1:

[0667] The server activates generative artificial intelligence to design and generate various customer interaction scenarios. These scenarios incorporate different customer attributes and create a list for the user to select from.

[0668] Step 2:

[0669] The terminal displays a list of scenarios sent from the server to the user. The user selects a scenario that suits their training needs and sends that selection from the terminal to the server.

[0670] Step 3:

[0671] The server configures the AI ​​character and interaction environment based on the selected scenario, and prepares to start the training session.

[0672] Step 4:

[0673] When a training session begins, the device activates its emotion engine, analyzing the user's facial expressions and voice in real time to acquire emotion data. This data is immediately sent to the server.

[0674] Step 5:

[0675] The user interacts with an AI-generated customer through their device. Based on emotional data provided by the emotion engine, the server generates the optimal response each time and adjusts the feedback provided to the user.

[0676] Step 6:

[0677] The server links user responses with emotional data, generates an AI response at the appropriate time, and sends it to the device for display to the user. This allows the user to continue interacting while receiving continuous feedback.

[0678] Step 7:

[0679] Once the training session is complete, the server comprehensively evaluates the user's interaction and emotional data to generate personalized feedback.

[0680] Step 8:

[0681] Feedback is provided to users through their devices and includes suggestions for improving customer service and advice on emotional responses.

[0682] Step 9:

[0683] The server records training history, evaluation information, and sentiment data in a database. Users can view this information on their devices and use it to improve their skills in the future.

[0684] (Example 2)

[0685] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0686] While customer service skills are becoming increasingly important in the service industry, traditional training methods struggle to improve these skills in realistic situations. In particular, there is a growing need to enhance the ability to respond to users' emotional states, but effective methods for achieving this are lacking. To address these issues, there is a need for the development of interactive training systems that analyze users' emotional states and incorporate the results.

[0687] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0688] In this invention, the server includes means for generating multiple dialogue scenarios using generative artificial intelligence, means for analyzing response data received from the user and creating an appropriate response using generative artificial intelligence, and means for analyzing the user's emotional state in real time using an emotion analysis engine and generating feedback based on that data. This makes it possible to provide a realistic simulation environment that takes the user's emotional state into account and to efficiently improve customer service skills.

[0689] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to automatically generate text, such as response scenarios, based on given prompts.

[0690] A "dialogue scenario" is a practice scenario that includes hypothetical conversation content and is used to simulate different customer attributes and situations.

[0691] "Response data" refers to the collection of reactions and input information that a user provides during an interaction.

[0692] An "emotion analysis engine" is a technology that evaluates and detects a user's emotional state in real time from data such as facial expressions and voice tone.

[0693] "Feedback" refers to information that includes evaluations and suggestions for improvement regarding the user's interactions, and is provided as part of training.

[0694] An "interface" is a means or method that provides a way for a user and a system to exchange information.

[0695] This invention is a training system for improving customer service skills in the service industry. The following describes a specific embodiment for carrying out the invention.

[0696] The server runs a generative artificial intelligence model on industry-standard computing equipment. This model utilizes generative AI models such as OpenAI's GPT series to generate customer service dialogue scenarios. An example prompt statement could be, "Generate dialogue scenarios that consider different customer attributes and situations." These generated scenarios will include a variety of customer attributes and response patterns.

[0697] The terminal is a device that functions as a user interface, providing a list of scenarios for the user to select. The terminal has a touchscreen display and audio input / output devices to accept user input. Based on the selected scenario, the terminal activates an emotion analysis engine and analyzes the user's emotional state in real time. This analysis is performed using EmotionAPI and other emotion recognition software. Based on data obtained from the user's facial expressions and tone of voice, the emotional state is quantified and sent to the server.

[0698] The server uses a generative AI model to generate optimal responses and feedback based on received sentiment data and user response data. This feedback includes specific advice on areas for improvement and skills to strengthen. This feedback, delivered through the user interface, allows users to improve their skills step-by-step and practically.

[0699] As a concrete example, during complaint handling training, generative artificial intelligence sets up a simulation role using the prompt, "Generate a conversation scenario that guides an angry customer to calm down." As the user attempts to handle the situation, the device recognizes emotions and continuously provides appropriate advice. This system allows users to efficiently improve their practical skills in customer service.

[0700] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0701] Step 1: Scenario Generation

[0702] The server activates a generative AI model and, using the configured prompt text as input, generates a variety of interaction scenarios. These scenarios include different customer attributes and situations. The output is a virtual dialogue scenario, which is saved in text format.

[0703] Step 2: Presenting the Scenario

[0704] The terminal receives scenarios output from the server and presents them to the user through a user interface. Specifically, it displays a list of scenarios on the terminal's display and allows selection via touch operation. The input is a list of scenarios, and the output is the selected scenario.

[0705] Step 3: Analysis of emotional data

[0706] The device activates an emotion analysis engine based on a scenario selected by the user. It analyzes the user's facial expressions and voice tone in real time and acquires this information as digital data. The input is the user's voice and video, and the output is quantified emotion data.

[0707] Step 4: Generating response feedback

[0708] The server receives emotion data and user responses from the terminal as input and uses a generative AI model to generate optimal feedback. The generated feedback includes areas for improvement and hints for better responses. The output is feedback in text format.

[0709] Step 5: Provide feedback

[0710] The terminal provides the user with feedback obtained from the server. Specifically, it can display advice on the terminal screen and also provide voice guidance. The input is text data of the feedback, and the output is the presentation of visual or auditory information to the user.

[0711] In this way, the system provides users with training in a realistic environment and helps improve their customer service skills based on emotion recognition.

[0712] (Application Example 2)

[0713] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0714] In the service industry, a challenge is providing real-time feedback to efficiently improve employees' customer service skills. Existing training systems lack the ability to handle diverse customer attributes and improve emotional care skills, and there is a lack of appropriate feedback that utilizes emotional information.

[0715] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0716] In this invention, the server includes means for generating multiple response scenarios using generative artificial intelligence, means for recognizing the user's emotional state in real time and optimizing the interaction using that information, and means for providing real-time feedback through an adaptive human interface. This enables users to efficiently improve their customer service skills.

[0717] "Generative artificial intelligence" is an artificial intelligence technology that generates responses based on user interaction and provides various response scenarios.

[0718] A "customer service scenario" is a hypothetical response case designed to simulate actual customer service situations and improve user skills.

[0719] "Emotional state" refers to the psychological and emotional condition that can be determined from the user's facial expressions and voice.

[0720] "Interaction" refers to the dynamic, two-way exchange that takes place between the user and the system.

[0721] An "adaptive human interface" is a point of contact between humans and machines that dynamically changes according to the user's situation to provide optimal feedback.

[0722] "Feedback" is a means of providing information to improve user performance based on the results of training sessions.

[0723] The system for realizing this invention primarily consists of a server, a terminal, and a user. This system is designed to improve customer service skills in the service industry. The server uses generative artificial intelligence to generate diverse response scenarios and provides them to the user. The user can access these scenarios through the terminal and implement appropriate responses.

[0724] The device uses an emotion engine to analyze the user's voice tone and facial expressions in real time. This emotion engine accurately captures the user's emotional state and transmits it to the server. This allows the server to generate responses and feedback tailored to the user's emotional state, providing a more realistic training environment. The server also maintains the user's training history and evaluation information to support long-term skill improvement.

[0725] As a concrete example, consider a scenario where a cafe employee practices handling customer complaints due to order errors. The server generates a scenario titled "Generating AI Model, Prompt Sentence," featuring a customer with rapidly changing emotions. The user selects this scenario and responds through interaction with the virtual customer. During this process, the terminal detects the user's emotional changes and provides appropriate feedback. For example, a prompt sentence such as "Please tell me how to respond when a customer is served the wrong drink" could be used. In this way, the system of the present invention enhances the user's ability to respond to emotional issues and helps them acquire better customer service skills.

[0726] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0727] Step 1:

[0728] The server generates diverse interaction scenarios using generative artificial intelligence. The server receives user training objectives and configuration parameters as input. Based on these inputs, the generative AI model generates scenarios with appropriate prompts and different customer attributes. The output consists of multiple virtual scenarios that the user can select from.

[0729] Step 2:

[0730] The user uses a terminal to select a response scenario provided by the server. During this selection process, the user interface displays an overview of each scenario. The terminal receives the user's selection information, and the output is a message indicating that the selected scenario is ready to begin.

[0731] Step 3:

[0732] The device uses an emotion engine to analyze the user's facial expressions and voice in real time. The input to the device consists of the user's raw voice data and facial image data. This data is processed using an emotion analysis algorithm to identify the user's emotional state. The output is the analyzed emotion data.

[0733] Step 4:

[0734] The server receives the analyzed emotion data and generates optimal feedback tailored to the user's emotional state. The server receives the result of the emotion analysis as input. Based on this data, the server utilizes a generation AI model to generate appropriate responses and follow-ups for the user. The output is a feedback message.

[0735] Step 5:

[0736] Feedback is provided to the user in real time via the device. Through this feedback, the user has the opportunity to adjust their response methods. The input to the device is a feedback message sent from the server, and the output is improvement guidelines provided to the user.

[0737] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0738] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0739] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0740] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0741] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0742] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0743] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0744] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0745] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0746] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0747] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0748] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0749] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0750] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0751] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0752] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0753] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0754] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0755] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0756] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0757] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0758] The following is further disclosed regarding the embodiments described above.

[0759] (Claim 1)

[0760] A means of generating multiple response scenarios using generative artificial intelligence,

[0761] A means for analyzing response data received from users and creating appropriate responses using generative artificial intelligence,

[0762] A means for conducting training sessions for users and evaluating the results of those sessions,

[0763] A means of providing feedback to users based on the evaluation results,

[0764] A system that includes means for retaining and managing training history and evaluation information.

[0765] (Claim 2)

[0766] The system according to claim 1, which simulates customer roles with different attributes using generative artificial intelligence during a training session, and enables user interaction based on these roles.

[0767] (Claim 3)

[0768] The system according to claim 1, which customizes user training content and optimizes response scenarios generated according to specified attributes.

[0769] "Example 1"

[0770] (Claim 1)

[0771] A means for generating dialogue scenarios with different conditions using generative artificial intelligence,

[0772] A means of analyzing data received from an information processing device and generating the optimal dialogue using generative artificial intelligence,

[0773] A means for conducting training sessions for users and determining the results of those sessions,

[0774] A means of presenting areas for improvement to users based on the assessment results,

[0775] A system that includes means for holding and managing training records and evaluation information.

[0776] (Claim 2)

[0777] The system according to claim 1, which, in a training session, uses generative artificial intelligence to mimic client roles with different characteristics and enables users to interact based on these roles.

[0778] (Claim 3)

[0779] The system according to claim 1, which adapts the user's training content and optimizes the generated dialogue scenario according to specified characteristics.

[0780] "Application Example 1"

[0781] (Claim 1)

[0782] A means of generating multiple response scenarios using generative artificial intelligence,

[0783] A means for analyzing response data received from users and creating appropriate responses using generative artificial intelligence,

[0784] A means for conducting training sessions for users and evaluating the results of those sessions,

[0785] A means of providing feedback to users based on the evaluation results,

[0786] A means of retaining and managing training history and evaluation information,

[0787] A means of providing a virtual dialogue to a user in real time via a visual interface device,

[0788] Means for providing real-time feedback through a visual interface device

[0789] A system that includes this.

[0790] (Claim 2)

[0791] The system according to claim 1, which simulates customer roles with different attributes using generative artificial intelligence during a training session, and enables user interaction based on these roles.

[0792] (Claim 3)

[0793] The system according to claim 1, which customizes user training content and optimizes response scenarios generated according to specified attributes.

[0794] "Example 2 of combining an emotion engine"

[0795] (Claim 1)

[0796] A means for generating multiple dialogue scenarios using generative artificial intelligence,

[0797] A means for analyzing response data received from users and generating appropriate responses using generative artificial intelligence,

[0798] A means for conducting training sessions for users and evaluating the results of those sessions,

[0799] A means of providing feedback to users based on the evaluation results,

[0800] Means for retaining and managing training history and evaluation information,

[0801] A means of analyzing a user's emotional state in real time using an emotion analysis engine and generating feedback based on that data,

[0802] Means for providing a user interface for presenting generated feedback

[0803] A system that includes this.

[0804] (Claim 2)

[0805] The system according to claim 1, which simulates user roles with different attributes using generative artificial intelligence during a training session, and enables user interaction based on these roles.

[0806] (Claim 3)

[0807] The system according to claim 1, which customizes the user's training content and optimizes the dialogue scenarios generated according to specified attributes.

[0808] "Application example 2 of combining emotional engines"

[0809] (Claim 1)

[0810] A means of generating multiple response scenarios using generative artificial intelligence,

[0811] A means for analyzing response data received from users and creating appropriate responses using generative artificial intelligence,

[0812] A means for conducting training sessions for users and evaluating the results of those sessions,

[0813] A means of providing feedback to users based on the evaluation results,

[0814] A means of retaining and managing training history and evaluation information,

[0815] A means of recognizing the user's emotional state in real time and using that information to optimize interaction,

[0816] A system that includes means of providing real-time feedback through an adaptive human interface.

[0817] (Claim 2)

[0818] The system according to claim 1, comprising means for enabling real-time interaction with customer roles having different attributes simulated by generative artificial intelligence.

[0819] (Claim 3)

[0820] The system according to claim 1, comprising means for optimizing a training session by analyzing the user's emotional state and customizing corresponding feedback. [Explanation of symbols]

[0821] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for generating multiple response scenarios using generative artificial intelligence, A means for analyzing response data received from users and creating appropriate responses using generative artificial intelligence, A means for conducting training sessions for users and evaluating the results of those sessions, A means of providing feedback to users based on the evaluation results, A system that includes means for retaining and managing training history and evaluation information.

2. The system according to claim 1, which simulates customer roles with different attributes using generative artificial intelligence during a training session, and enables user interaction based on these roles.

3. The system according to claim 1, which customizes user training content and optimizes response scenarios generated according to specified attributes.

Images