System

Abstract

A system is provided.SOLUTION: A system comprising: means for recording Talkscripts and know-how of high productivity workers; means for pre-processing the recorded Talkscripts and know-how; means for training a generative artificial intelligence with the pre-processed data; means for evaluating and improving the performance of a generative artificial intelligence model; means for creating and distributing high performance Talkscripts to workers; and means for collecting feedback from workers to further improve the generative artificial intelligence model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In customer service work, skill differences between highly productive workers and other workers can lead to variations in the quality of service provided and performance. This creates a situation where the work relies solely on a few workers, hindering overall productivity. Furthermore, conventional methods for creating and training talk scripts often rely on "gut feeling," making it difficult to generate and standardize effective, theory-based scripts. This invention aims to solve these problems and ensure that all workers have uniform, high-level customer service skills. [Means for solving the problem]

[0005] The present invention provides the following means. First, a means for recording the talk scripts and know-how of highly productive workers is provided. Next, a means for preprocessing the recorded talk scripts and know-how is used to clean the data and standardize the format. Then, a means for training the preprocessed data using a generative artificial intelligence is employed. A means for evaluating the performance of the generative artificial intelligence model and improving it based on feedback is provided. This also includes a means for creating high-performance talk scripts and distributing them to all workers. Finally, by adding a means for collecting feedback from workers after training sessions and further improving the generative artificial intelligence model, all workers are able to possess uniformly high levels of customer service skills.

[0006] A "highly productive worker" is a worker who can produce higher results and be more efficient than a standard worker.

[0007] A "talk script" is a collection of standardized conversation examples and responses used by workers when dealing with customers or serving them.

[0008] "Know-how" is technical knowledge and methods gained from practical experience in a specific business or task.

[0009] "Generative AI" is an AI technology that can generate new data and results based on collected data.

[0010] "Preprocessing" refers to the cleaning and transformation of data to make it suitable for learning and analysis.

[0011] "Learning" is the process by which generative artificial intelligence finds patterns and rules based on the data provided and builds them into a model.

[0012] "Evaluating performance" means measuring how accurately and effectively a trained generative artificial intelligence model can achieve its goals.

[0013] "Feedback" refers to reactions and opinions received about a system or process, especially information that includes areas for improvement or positive aspects.

[0014] A "training session" is an opportunity for a worker to practice or gain new skills or knowledge. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] MODE FOR CARRYING OUT THE INVENTION

[0037] The present invention provides a system that collects customer service scripts and know-how from highly productive workers and trains them using generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills. An embodiment of the present invention is described in detail below.

[0038] Data collection

[0039] First, the user (high productivity worker) records voice and text data on the terminal while serving customers. The recorded voice data is the content of the conversation and response during the customer service. The user (high productivity worker) then uploads the recorded data to the server via a dedicated web interface.

[0040] Data Preprocessing

[0041] The uploaded data is preprocessed by the server, which converts the audio data into text data, removes unnecessary noise and irrelevant information, and converts different text formats into a unified format, ensuring data consistency.

[0042] Learning Process

[0043] The preprocessed data is fed to a generative artificial intelligence model. The server then executes a learning process based on the data, teaching the model the specific skills and effective speech scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized.

[0044] Model evaluation and improvement

[0045] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and, if there are any shortcomings, determines that the model needs to be retrained.

[0046] Script generation and distribution

[0047] After generating a high-performance talk script, the server distributes it to all workers. Specifically, users (workers) use the distributed talk script to conduct training sessions. In these sessions, role-playing using the new script is conducted, simulating the application of the script to real customer service work.

[0048] Gathering feedback and relearning

[0049] After the training session, the user (worker) records feedback through a dedicated interface. The server analyzes the collected feedback and performs re-training to improve the model based on the feedback. This allows the generative artificial intelligence model to be continuously improved.

[0050] Specific examples

[0051] Example 1: Collecting and learning talk scripts

[0052] 1. The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving the customer. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[0053] 2. The user uploads the recorded data to the server through a web interface.

[0054] 3. The server converts the uploaded audio data into text data and removes unnecessary noise.

[0055] 4. The preprocessed data is fed into the generative artificial intelligence, which begins learning.

[0056] Example 2: Training and feedback

[0057] 1. The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[0058] 2. The user (operator) conducts a training session using the distributed script.

[0059] 3. After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[0060] 4. The server analyzes the collected feedback and retrains the model if necessary.

[0061] This invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. Furthermore, through continuous feedback and re-learning, the generative AI model can continue to provide the latest, high-quality scripts.

[0062] The processing flow will be explained below.

[0063] Program processing flow

[0064] Step 1: Data collection

[0065] 1. User (highly productive worker)

[0066] Record audio of talk scripts used by highly productive workers in their daily customer service work.

[0067] For example: "Hello, thank you for visiting. How can I help you today?"

[0068] 2. Terminal

[0069] Stores recorded audio data and generates a corresponding text log.

[0070] Saving example: audio_dataA.wav, text_logA.txt

[0071] 3. User (highly productive worker)

[0072] The collected data is uploaded to a server via a dedicated web interface.

[0073] Upload example: Send text_logA.txt to the server.

[0074] Step 2: Data Preprocessing

[0075] 1. Server

[0076] Convert the uploaded voice data into text data (voice recognition processing).

[0077] Example of converted text: audio_dataA.wav → "Hello, and welcome to SoftBank. How can we help you today?"

[0078] 2. Server

[0079] Preprocessing is performed to standardize the converted text data into a uniform format.

[0080] Preprocessing example: Removing unnecessary symbols and noise data.

[0081] 3. Server

[0082] The preprocessed data is formatted as a talk script.

[0083] Formatting example: "Hello, welcome to SoftBank. What can I do for you today?"

[0084] Step 3: Model training

[0085] 1. Server

[0086] The formatted talk script is fed into a generative artificial intelligence model to begin the learning process.

[0087] Example learning: "Calling customers by name can lead to more effective customer service."

[0088] 2. Server

[0089] Iteratively train the data until the model is trained.

[0090] Example iteration: Run training for 100 epochs.

[0091] Step 4: Model evaluation

[0092] 1. Server

[0093] A separate validation dataset is prepared to evaluate the performance of the trained model, and validation is performed.

[0094] Validation example: Analysis of accuracy, customer satisfaction scores.

[0095] 2. User (Administrator)

[0096] Check the evaluation results of the trained model and determine if there are any deficiencies and whether the model needs to be retrained.

[0097] Example: If the assessment score is less than 95%, decide to restudy.

[0098] Step 5: Generate and distribute scripts

[0099] 1. Server

[0100] High-performance talk scripts are generated based on successful models and distributed to all workers.

[0101] Example of a distribution script: "Thank you for visiting us, Mr. / Ms. XX. What kind of service would you like today?"

[0102] 2. Terminal

[0103] Set up a training session for all workers and role-play using the new script.

[0104] Step 6: Gather feedback and improve

[0105] 1. User (operator)

[0106] Record feedback after training sessions using a dedicated interface.

[0107] Example feedback: "Customers responded very well to the new script, but often prefer a more casual tone."

[0108] 2. Server

[0109] Analyze the collected feedback, identify areas for improvement in the model, and retrain it.

[0110] Example of improvement: Use the collected feedback to further casualize the script.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] In conventional customer service operations, it was difficult to effectively share the talk scripts and know-how of highly productive workers with other workers. As a result, there was a large difference in customer service skills among individuals, resulting in an inconsistent overall service quality. Furthermore, the process of improving customer service scripts through continuous training and feedback was not carried out efficiently, making it difficult to achieve sustained operational improvement. A system was needed to solve these issues and enable all workers to acquire and maintain a high level of customer service skills.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes: means for recording the talk scripts and know-how of highly productive workers; means for uploading the recorded talk scripts and know-how to the server through a dedicated interface; means for converting the uploaded voice data into text data and removing unnecessary noise and irrelevant information; means for converting the preprocessed data into a unified format; means for training a generative artificial intelligence (AI) based on the preprocessed data; means for evaluating the performance of the AI ​​model using a separate validation dataset and identifying areas for improvement; means for generating high-performance talk scripts and distributing them to all workers; means for workers to record feedback after training sessions; and means for analyzing the collected feedback and retraining the AI ​​model to continuously improve it. This enables all workers to have a uniformly high level of customer service skills, facilitating continuous improvement and improving overall service quality.

[0116] A "highly productive worker" is a worker who has outstanding skills and know-how in customer service work and demonstrates higher productivity and effectiveness than other workers.

[0117] A "talk script" is a set of predetermined conversational content and phrases used during customer service.

[0118] "Know-how" refers to practical knowledge based on specialized knowledge, skills, and experience required to effectively carry out specific tasks or operations.

[0119] A "dedicated interface" is a specific operating screen or system that allows users to input, manage, and transmit data.

[0120] A "server" is a computer system that stores and processes data via a network, and plays a central role in this system.

[0121] "Preprocessing" refers to a series of processing operations performed on raw data to prepare it in a form suitable for analysis and learning.

[0122] "Generative AI" is a type of AI that has the ability to learn patterns from large amounts of data and generate and respond to new data.

[0123] "High-performance talk scripts" refer to standard, high-quality conversation phrases and content generated by generative artificial intelligence.

[0124] "Feedback" refers to the evaluations and opinions provided by workers after training sessions or actual work performance, which are used to improve the system.

[0125] A "training session" is a training activity in which workers simulate real-world work scenarios using pre-prepared scripts.

[0126] "Retraining" is the process of retraining a generative artificial intelligence model based on new data and feedback collected to improve its performance.

[0127] "Audio data" refers to data that records the user's voice, and is used to record the contents of the conversation.

[0128] "Text data" is voice data converted into text information and is used for analysis and learning.

[0129] "Noise" refers to unnecessary information or noise that should not be included in the data being analyzed.

[0130] A "format" refers to the form or structure that allows data to be handled uniformly.

[0131] A "validation dataset" is data prepared separately from the training data and used to evaluate the performance of a generative artificial intelligence model.

[0132] MODE FOR CARRYING OUT THE INVENTION

[0133] The present invention provides a system that collects customer service scripts and know-how from highly productive workers and trains them in a generative artificial intelligence (AI) system, thereby enabling all workers to have uniformly high levels of customer service skills. The following describes an embodiment of the present invention.

[0134] Data collection

[0135] The user (highly productive worker) records voice and text data during customer service operations on a terminal. This terminal is a device with a recording function, such as a smartphone or tablet. The recorded voice data specifically includes the content of the conversation during customer service, such as "Hello, thank you for visiting our store. What can I do for you today?" The user uploads the recorded data to a server via a dedicated web interface.

[0136] Data Preprocessing

[0137] The server converts the uploaded voice data into text data using speech recognition software such as the Google Cloud Speech-to-Text API. The server then uses text analysis tools such as the NLTK library to remove unnecessary noise and irrelevant information. This ensures data consistency and converts it into a unified format (e.g., JSON).

[0138] Learning Process

[0139] The preprocessed data is fed into a generative artificial intelligence model (e.g., OpenAI's GPT-3). The server executes the learning process based on the specified number of epochs (e.g., 100 epochs). During the learning process, the model learns the specific skills and effective talking scripts of highly productive workers. This process is distributed using a high-performance GPU cluster.

[0140] Model evaluation and improvement

[0141] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. Metrics such as F1 score and accuracy are used in the evaluation. The user (administrator) can check the evaluation results through the dashboard and decide whether to retrain the model if there are any shortcomings.

[0142] Script generation and distribution

[0143] Once a high-performance talk script has been generated, the server distributes it to all workers. Specifically, the server sends notifications via email or a dedicated app. For example, a talk script such as "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?" may be distributed. Users (workers) use the distributed script to conduct training sessions and role-play.

[0144] Gathering feedback and relearning

[0145] After the training session, the user (worker) records their impressions and suggestions for improvement through a dedicated feedback interface (e.g., a web form). For example, specific opinions such as "The new script was very effective, but I thought it needed to be a little more casual in its expressions" can be recorded. The server analyzes the collected feedback using natural language processing technology and reflects it in the next re-training. This allows the generative AI model to be continuously improved.

[0146] Specific examples

[0147] Example 1: Collecting and learning talk scripts

[0148] The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving customers. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[0149] The user uploads the recorded data to a server through a web interface.

[0150] The server converts the uploaded voice data into text data and removes unnecessary noise.

[0151] The pre-processed data is fed into the generative artificial intelligence, which begins learning.

[0152] Example 2: Training and feedback

[0153] The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[0154] The user (operator) conducts a training session using the distributed script.

[0155] After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[0156] The server analyzes the collected feedback and retrains the model if necessary.

[0157] Prompt Sentence Examples

[0158] "We want to train the AI ​​based on customer service scripts written by highly productive workers. We would like you to record the following conversations and carry out the training process:

[0159] "Hello, thank you for coming. How can I help you today?"

[0160] "Thank you for visiting us, Mr. / Ms. X. What kind of service would you like today?"

[0161] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0162] Step 1: Data collection

[0163] The user (highly productive worker) records conversations during customer service duties on a device. For example, using a recording app on a smartphone, the user can record a conversation that goes something like, "Hello, thank you for visiting our store. What can I do for you today?" Once the recording is complete, the user uploads the data to a server via a dedicated web interface.

[0164] Input: Voice data during customer service

[0165] Output: Recording data is uploaded to the server

[0166] How it works: Record audio using a recording app on your smartphone and upload the file to a server using a dedicated web interface. The web interface is an application that runs on a browser.

[0167] Step 2: Data Preprocessing

[0168] The server converts the uploaded audio data into text data. It uses speech recognition software, such as the Google Cloud Speech-to-Text API, to convert the recording into text. The server then uses a text analysis tool, such as NLTK, to remove unnecessary noise and irrelevant information from the data. It also converts the text data into a standardized format, such as JSON.

[0169] Input: Uploaded audio data

[0170] Output: Preprocessed text data

[0171] What it does: The server runs speech recognition software, analyzes the resulting text data, removes noise, and converts the data into a unified format.

[0172] Step 3: The learning process

[0173] The server supplies the preprocessed text data to a generative artificial intelligence model (e.g., OpenAI's GPT-3). The learning process begins based on a specified number of epochs (e.g., 100 epochs). The model learns the specific skills and speech scripts of highly productive workers. This process is distributed using a high-performance GPU cluster.

[0174] Input: Preprocessed text data

[0175] Output: A trained generative artificial intelligence model

[0176] Specific operation: The server inputs text data into the generative AI model and performs learning for the specified number of epochs. The learning progress is recorded in the server's log.

[0177] Step 4: Model evaluation and improvement

[0178] The server evaluates the performance of the generative AI model using a separate validation dataset. The effectiveness of the model is confirmed based on evaluation indicators such as F1 score and accuracy. The user (administrator) can check the evaluation results via the dashboard and decide whether to retrain the model if any shortcomings are found.

[0179] Input: trained model, validation dataset

[0180] Output: Model performance evaluation results

[0181] Specific operation: The server inputs the validation data into the model and displays the obtained score on the dashboard. The administrator can check the results on the dashboard and instruct the model to be retrained if necessary.

[0182] Step 5: Generate and distribute scripts

[0183] The server generates high-performance conversation scripts and distributes them to all workers. Specifically, the server sends notifications via email or a dedicated app. For example, a conversation script such as "Thank you for visiting our store, Mr. / Ms. X. What kind of service would you like today?" is generated and distributed. Users (workers) use the distributed scripts to conduct training sessions and role-play.

[0184] Input: A trained model

[0185] Output: Generated talk script

[0186] How it works: The server uses the trained model to generate a talk script and sends it to the worker via email or a dedicated app. The worker receives the script and practices it during the training session.

[0187] Step 6: Gather feedback and retrain

[0188] After the training session, the user (worker) records their impressions and suggestions for improvement through a dedicated feedback interface (e.g., a web form). The server analyzes the collected feedback using natural language processing technology and continuously improves the generative AI model by retraining it.

[0189] Input: Feedback data

[0190] Output: An improved generative artificial intelligence model

[0191] Specific operation: The worker enters feedback into a web form and submits it. The server analyzes the feedback data and retrains the model as necessary. The retrained model is used to generate a new talk script.

[0192] (Application example 1)

[0193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] In conventional customer service work, there was a large variation in the customer service skills and experience of individual workers, resulting in uneven customer satisfaction and work efficiency. Furthermore, it took time and effort for workers to learn new customer service scripts, making it difficult to consistently provide high-quality service. Furthermore, there was no system in place to provide effective customer service phrases in real time, which sometimes led to inappropriate responses during customer service.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0196] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for collecting feedback from workers and further improving the AI ​​model, and means for the AI ​​to suggest customer service phrases in real time when a store clerk converses with a customer. This allows all workers to have uniformly high levels of customer service skills and enables the provision of effective customer service phrases in real time.

[0197] "Highly productive workers" refer to workers who have high productivity and excellent skills in customer service work.

[0198] A "talk script" refers to a series of phrases and responses used in customer service.

[0199] "Know-how" refers to techniques, knowledge, and methods that are effective in a specific business or task.

[0200] "Preprocessing" refers to the process of removing unnecessary noise and irrelevant information from recorded data and standardizing it into an appropriate format.

[0201] "Generative AI" refers to AI that has the ability to create new data and scripts based on input data.

[0202] "Performance evaluation of a generative artificial intelligence model" refers to the act of verifying whether the output of a generative artificial intelligence model reaches the expected level.

[0203] A "high-performance talk script" refers to a series of effective, high-quality customer service phrases optimized using generative artificial intelligence.

[0204] "Means of suggesting conversation phrases in real time" refers to a system in which generative artificial intelligence instantly suggests the most appropriate phrases when a store clerk is conversing with a customer.

[0205] The present invention is a system that collects customer service scripts and know-how from highly productive workers, trains them into a generative artificial intelligence model, and enables all workers to acquire high-level customer service skills. Specific embodiments for carrying out the invention are described below.

[0206] Data collection

[0207] First, the user (highly productive worker) records conversations during customer service using the recording function of their smartphone. The user then uploads the recorded audio data to a server via a dedicated web interface. The audio data includes the conversations and responses made during customer service.

[0208] Data Preprocessing

[0209] Once the server receives the uploaded audio data, it converts it into text using the SpeechRecognition library. It then removes unnecessary noise and irrelevant information and organizes the text data into a unified format. This preprocessing process ensures consistent data suitable for analysis.

[0210] Learning Process

[0211] The preprocessed data is fed to a generative AI model (e.g., GPT-4). The server then executes a model learning process based on this data, teaching the model the specific skills and effective speech scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized.

[0212] Model evaluation and improvement

[0213] After the training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model, and the user (administrator) can review the evaluation results and determine whether the model needs to be retrained if there are any shortcomings.

[0214] Script generation and distribution

[0215] After generating a high-performance talk script, the server distributes it to all workers. The users (workers) then use the distributed talk script to conduct training sessions. In these sessions, role-playing using the new script is conducted, simulating the application of the script to real customer service tasks.

[0216] Real-time phrase suggestions

[0217] During customer service, the server uses generative artificial intelligence to suggest optimal response phrases in real time. The user (worker) receives these in real time on their smartphone and uses them in conversation with the customer. This enables high-quality responses that are immediately suited to the situation.

[0218] Gathering feedback and relearning

[0219] After the training session, the user (worker) records feedback using a dedicated interface, and the server analyzes this feedback and performs re-training to further improve the generative artificial intelligence model.

[0220] Specific examples

[0221] For example, imagine a scenario where a new sales associate greets a customer with, "Hello, thank you for your inquiry. How can I help you today?" This conversation is recorded on a smartphone and uploaded through an application. The generative AI model analyzes this data and suggests an optimized script.

[0222] Example prompts for generative AI models

[0223] An example prompt is:

[0224] Prompt: Please optimize the following customer service script.

[0225] input:

[0226] "Hello, thank you for coming. How can I help you today?"

[0227] the goal:

[0228] 1. Address customers in a bright, friendly tone.

[0229] 2. Include questions to quickly understand the customer's needs.

[0230] Example output:

[0231] "Hello! Welcome. How can I help you today?"

[0232] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0233] Step 1:

[0234] The user (highly productive worker) records conversations while serving customers using their smartphone.

[0235] Input: Recording of voice conversations using a smartphone

[0236] Output: Recorded audio data file (e.g. .wav)

[0237] How it works: Use a dedicated application on your smartphone to record audio data through the microphone. Once the recording is complete, press the save button to create a data file.

[0238] Step 2:

[0239] Users upload the recorded audio data to the server via a dedicated web interface.

[0240] Input: Recorded audio data file

[0241] Output: Audio data file stored on the server

[0242] Specific operations: Access the web interface, press the upload button to select a file, and press the send button to transfer the audio data to the server.

[0243] Step 3:

[0244] The server receives the uploaded voice data and converts it into text data using the SpeechRecognition library.

[0245] Input: Uploaded audio data file

[0246] Output: Text data converted from audio data

[0247] Specific operation: Analyzes the audio file using the SpeechRecognition library and converts the audio signal into a string. The converted text data is temporarily saved.

[0248] Step 4:

[0249] The server preprocesses the text data, removing unnecessary noise and irrelevant information, and arranging it into a unified format.

[0250] Input: Converted text data

[0251] Output: Preprocessed clean text data

[0252] Specific operation: Using noise filtering algorithms and regular expressions, unnecessary parts are removed from text data and it is formatted into a unified format.

[0253] Step 5:

[0254] The pre-processed data is fed into a generative artificial intelligence model to begin the learning process.

[0255] Input: Preprocessed text data

[0256] Output: Trained generative AI model (initial version)

[0257] What it does: It feeds data to a generative AI model (e.g., GPT-4) and trains it for a specified number of epochs. The model learns based on a specific conversation script and response style.

[0258] Step 6:

[0259] The server evaluates the model using a separate validation dataset to evaluate the performance of the trained model.

[0260] Input: trained model and validation dataset

[0261] Output: Model performance evaluation results (accuracy, effectiveness, etc.)

[0262] What it does: It evaluates the model output using a validation dataset to measure accuracy and effectiveness, and stores the results in a database.

[0263] Step 7:

[0264] The user (administrator) checks the evaluation results and, if there are any shortcomings, instructs the model to be retrained.

[0265] Input: Model performance evaluation results

[0266] Output: Retraining instructions or model approval

[0267] Specific behavior: The administrator reviews the assessment results and determines whether retraining is necessary. If necessary, the administrator triggers the retraining process.

[0268] Step 8:

[0269] After the high-performance talk script is generated, the server distributes it to all workers.

[0270] Input: The final trained generative artificial intelligence model

[0271] Output: Distributed high performance talk script

[0272] Specific operation: The server sends the script to each worker's terminal and instructs them to conduct a training session using the new script.

[0273] Step 9:

[0274] After the training session, the user (worker) records feedback using a dedicated interface.

[0275] Input: Operator feedback during training session

[0276] Output: Recorded feedback data

[0277] What it does: Workers complete a training session and then use a dedicated web interface to provide feedback on the effectiveness of the script.

[0278] Step 10:

[0279] The server analyzes the collected feedback and performs re-training to further improve the generative artificial intelligence model.

[0280] Input: Recorded feedback data

[0281] Output: An improved generative artificial intelligence model

[0282] What it does: Update the generative AI model based on the feedback data, then run another epoch of training, reassessing the model's accuracy and effectiveness, and continuously optimizing it.

[0283] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0284] MODE FOR CARRYING OUT THE INVENTION

[0285] The present invention provides a system that collects the customer service scripts and know-how of highly productive workers and trains them into a generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills, and also combines it with an emotion engine that recognizes the emotions of users.Embodiments of the present invention are described in detail below.

[0286] Data collection

[0287] First, the user (high productivity worker) records voice and text data on the terminal while serving customers. The recorded voice data is the content of the conversation and response during the customer service. The user (high productivity worker) then uploads the recorded data to the server via a dedicated web interface.

[0288] Data Preprocessing

[0289] The uploaded data is preprocessed by the server, which converts the audio data into text data, removes unnecessary noise and irrelevant information, and converts different text formats into a unified format, ensuring data consistency.

[0290] emotion recognition

[0291] The acquired voice data may contain the user's emotional information. The terminal uses an emotion engine to recognize the user's emotion from the data. The emotion recognition result is used in subsequent processing.

[0292] Learning Process

[0293] The preprocessed data is supplied to the generative AI model. The server then executes a learning process based on this data, teaching the model the specific skills and effective conversation scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is fully optimized. The emotional information recognized by the emotion engine can also be taught to the generative AI model, enabling more human-like responses.

[0294] Model evaluation and improvement

[0295] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and, if there are any shortcomings, determines that the model needs to be retrained.

[0296] Script generation and dynamic adjustment

[0297] Once a high-performance talk script is generated, it is distributed to all workers. The talk script is dynamically adjusted according to the user's emotions. Specifically, the server recognizes the user's emotions in real time using an emotion engine and adjusts the talk script based on that information. For example, if the user is dissatisfied, a script with a more polite and comforting tone is used.

[0298] Gathering feedback and relearning

[0299] After the training session, the user (worker) records feedback through a dedicated interface. The server analyzes the collected feedback and performs re-training to improve the model based on the feedback. This allows the generative artificial intelligence model to be continuously improved.

[0300] Specific examples

[0301] Example 1: Collecting and learning talk scripts

[0302] 1. The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving the customer. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[0303] 2. The user uploads the recorded data to the server through a web interface.

[0304] 3. The server converts the uploaded audio data into text data and removes unnecessary noise.

[0305] 4. The preprocessed data is fed to the generative AI, where learning begins. The emotion engine also extracts the user's emotional information from the voice data, which is then used as learning data.

[0306] Example 2: Dynamic talk script generation and feedback

[0307] 1. The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[0308] 2. The user (worker) conducts a training session using the distributed script. During the training, the emotion engine recognizes the user's emotions in real time and adjusts the script accordingly.

[0309] 3. After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[0310] 4. The server analyzes the collected feedback and retrains the model if necessary.

[0311] In this way, the system of the present invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. In addition, the introduction of an emotion engine enables more personalized customer service, which will also improve customer satisfaction.

[0312] The processing flow will be explained below.

[0313] Program processing flow

[0314] Step 1: Data collection

[0315] 1. User (highly productive worker)

[0316] Record audio of talk scripts used by highly productive workers in their daily customer service work.

[0317] For example: "Hello, thank you for visiting. How can I help you today?"

[0318] 2. Terminal

[0319] Stores recorded audio data and generates a corresponding text log.

[0320] Saving example: audio_dataA.wav, text_logA.txt

[0321] 3. User (highly productive worker)

[0322] The collected data is uploaded to a server via a dedicated web interface.

[0323] Upload example: Send text_logA.txt to the server.

[0324] Step 2: Data Preprocessing

[0325] 1. Server

[0326] Convert the uploaded voice data into text data (voice recognition processing).

[0327] Example of converted text: audio_dataA.wav → "Hello, thank you for visiting. How can we help you today?"

[0328] 2. Server

[0329] Preprocessing is performed to standardize the converted text data into a uniform format.

[0330] Preprocessing example: Removing unnecessary symbols and noise data.

[0331] 3. Server

[0332] The preprocessed data is formatted as a talk script.

[0333] Formatting example: "Hello, thank you for visiting. What are you looking for today?"

[0334] Step 3: Emotion Recognition

[0335] 1. Terminal

[0336] The acquired voice data is analyzed by an emotion engine to identify the user's emotions.

[0337] Analysis example: Identifying emotions such as "joy," "anger," "sadness," and "surprise" from voice.

[0338] 2. Terminal

[0339] The identified emotion information is transmitted to a server together with the text data.

[0340] Example of transmission: Emotion information "joy" is sent to the server.

[0341] Step 4: Model training

[0342] 1. Server

[0343] The formatted talk script and the emotional information obtained from the emotion engine are fed into the generative artificial intelligence model to begin the learning process.

[0344] Example learning: "Calling customers by name can lead to more effective customer service."

[0345] 2. Server

[0346] Iteratively train the data until the model is trained.

[0347] Example iteration: Run training for 100 epochs.

[0348] Step 5: Model evaluation

[0349] 1. Server

[0350] A separate validation dataset is prepared to evaluate the performance of the trained model, and validation is performed.

[0351] Validation example: Analysis of accuracy, customer satisfaction scores.

[0352] 2. User (Administrator)

[0353] Check the evaluation results of the trained model and determine if there are any deficiencies and whether the model needs to be retrained.

[0354] Example: If the assessment score is less than 95%, decide to restudy.

[0355] Step 6: Generate and distribute scripts

[0356] 1. Server

[0357] High-performance talk scripts are generated based on successful models and distributed to all workers.

[0358] Example of a distribution script: "Thank you for visiting us, Mr. / Ms. XX. What kind of service would you like today?"

[0359] 2. Terminal

[0360] Set up a training session for all workers and role-play using the new script.

[0361] Step 7: Dynamic Adjustment

[0362] 1. Terminal

[0363] During a training session, an emotion engine is used in real time to recognize the user's emotions.

[0364] Analysis example: If the user feels "unhappy" during training, a corresponding script is recommended.

[0365] 2. Server

[0366] The talk script is dynamically adjusted based on emotional information obtained in real time.

[0367] Example of adjustment: If the user is perceived as "unhappy," change the talk script to a more polite and comforting tone.

[0368] Step 8: Gather feedback and improve

[0369] 1. User (operator)

[0370] Record feedback after training sessions using a dedicated interface.

[0371] Example feedback: "Customers responded very well to the new script, but often prefer a more casual tone."

[0372] 2. Server

[0373] Analyze the collected feedback, identify areas for improvement in the model, and retrain it.

[0374] Example of improvement: Use the collected feedback to further casualize the script.

[0375] In this way, the system of the present invention standardizes the quality of customer service scripts and improves overall productivity. Furthermore, the introduction of an emotion engine enables more personalized customer service, thereby improving customer satisfaction.

[0376] Example 2

[0377] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0378] It is necessary to uniformly impart the skills and know-how of highly productive workers to all workers, so that they can have a high level of customer service skills. There is also a need to improve customer satisfaction by recognizing users' emotions in real time and providing personalized responses. As it is difficult to effectively solve these issues using conventional methods, a new system is needed.

[0379] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0380] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for collecting feedback from workers and further improving the AI ​​model, and means for recognizing user emotions and dynamically adjusting the talk scripts based on that information. This enables all workers to have uniformly high levels of customer service skills and realizes personalized responses according to the user's emotions.

[0381] A "highly productive worker" refers to a worker who has excellent customer service skills and efficiency, and who achieves excellent results in their work.

[0382] A "talk script" refers to a document or data that compiles standard conversation flows and phrases used when serving or interacting with customers.

[0383] "Know-how" refers to practical knowledge and skills for performing specific tasks or operations.

[0384] "Preprocessing" refers to the process of removing noise and standardizing the format of data before it is fed into a generative artificial intelligence model.

[0385] "Generative AI" refers to AI that generates new data and responses based on learned data.

[0386] A "generative artificial intelligence model" refers to an algorithm or structure for prediction or generation that is constructed based on the learning results of generative artificial intelligence.

[0387] "Evaluating and improving performance" refers to verifying the accuracy and effectiveness of the generative AI model, and retraining or adjusting the model as necessary.

[0388] "High-performance talk scripts" refer to talk scripts generated by a generative artificial intelligence model that are evaluated as having high customer service skills and effectiveness.

[0389] "Feedback" refers to collecting opinions and impressions that workers gain through training sessions and on-the-job tasks.

[0390] "Recognizing user emotions" refers to analyzing and acquiring the user's emotional state from voice and text data.

[0391] "Dynamic adjustment" refers to changing the content of the talk script provided in real time according to the user's situation and emotions.

[0392] MODE FOR CARRYING OUT THE INVENTION

[0393] The present invention provides a system that collects the customer service scripts and know-how of highly productive workers and trains them into a generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills, and also combines it with an emotion engine that recognizes the emotions of users.Embodiments of the present invention are described in detail below.

[0394] Data collection

[0395] First, the user (highly productive worker) records voice and text data on the device while serving customers. Specifically, the device's recording function is used to record the conversation. For example, a script such as "Hello, thank you for visiting. What can I do for you today?" is recorded. The recorded voice data and related text data are then uploaded to the server via a dedicated web interface. Using a web browser, the user authenticates on the login screen, clicks the "File Upload" button, selects the data, and then clicks the "Send" button to complete the upload.

[0396] Data Preprocessing

[0397] The uploaded data is preprocessed by the server. First, speech recognition software (e.g., Google Cloud Speech-to-Text API) is used to convert the audio data into text data. The server sends the audio file to the speech recognition API and receives the converted text data. Then, it removes unnecessary noise and irrelevant information from the text data and standardizes the data into a standardized format using natural language processing tools (e.g., Python's NLTK library). This generates data in a standardized format.

[0398] emotion recognition

[0399] The acquired voice data may contain the user's emotional information. The terminal uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotion from this data. Specifically, the terminal inputs the voice data into the emotion recognition engine and obtains the output emotion label. This allows the emotional information to be extracted.

[0400] Learning Process

[0401] The preprocessed data is fed into a generative AI model (for example, OpenAI's GPT model). The server then runs a learning process based on this data, teaching the model the specific skills and effective conversation scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized. The emotional information recognized by the emotion engine can also be taught to the generative AI model, enabling it to respond more like a human.

[0402] Model evaluation and improvement

[0403] After training is complete, the server evaluates the performance of the generative artificial intelligence model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and determines whether the model needs to be retrained if there are any shortcomings. The evaluation results are displayed visually in a web interface, and the administrator can start retraining the model by pressing the "Retrain" button.

[0404] Script generation and dynamic adjustment

[0405] Once a high-performance talk script has been generated, it is distributed to all workers. The talk script is dynamically adjusted according to the user's emotions. Specifically, the server recognizes the user's emotions in real time using an emotion recognition engine and adjusts the talk script based on that information. For example, a prompt such as "For dissatisfied users, generate a script with a more polite and comforting tone" can be input and the generated script can be obtained.

[0406] Gathering feedback and relearning

[0407] After the training session, the user (worker) records feedback through a dedicated interface. For example, "The new script was very effective, but I thought it needed to be a little more casual." The server analyzes the collected feedback and retrains the model as needed. This allows the generative artificial intelligence model to continuously improve.

[0408] In this way, the system of the present invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. Furthermore, the introduction of an emotion engine enables more personalized customer service, which will also improve customer satisfaction.

[0409] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0410] Step 1:

[0411] The user records the conversation during the customer service session using the device's recording function. Specifically, the user presses the recording button on the device to start the conversation, and presses the button again to stop recording when finished. The input is audio data, and the output is an audio file saved on the device.

[0412] Step 2:

[0413] The user uploads the recorded audio data and associated text data to the server through a dedicated web interface. Specifically, the user opens a web browser, authenticates, clicks the "File Upload" button, selects the audio file, and clicks the "Send" button. The input is the audio file and text data, and the output is the data uploaded to the server.

[0414] Step 3:

[0415] The server converts the uploaded voice data into text data using voice recognition software (e.g., Google Cloud Speech-to-Text API). Specifically, the server sends the voice file to the API and receives the converted text data. The input is voice data, and the output is text data.

[0416] Step 4:

[0417] The server uses natural language processing tools (e.g., Python's NLTK library) to remove unnecessary noise from the converted text data and standardize the data into a format. Specifically, it analyzes the text data, removes unnecessary words and sentences, and converts them into grammatically correct sentences. The input is text data, and the output is formatted text data.

[0418] Step 5:

[0419] The device uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotion from the recorded voice data. Specifically, the voice data is input to the engine, and an emotion label is output. The input is the voice data, and the output is emotion information.

[0420] Step 6:

[0421] The server feeds the preprocessed text data and sentiment information to a generative artificial intelligence model (e.g., OpenAI's GPT model). The server inputs the dataset into the model's training script and runs training for a specified number of epochs. The input is the preprocessed data, and the output is a trained model.

[0422] Step 7:

[0423] The server uses a separate validation dataset to evaluate the performance of the trained model. Specifically, it inputs test data to the model and runs a script that calculates evaluation metrics such as precision and recall. The input is the validation dataset, and the output is the evaluation results.

[0424] Step 8:

[0425] The user (administrator) checks the evaluation results and decides whether to retrain if necessary. The evaluation results are visually displayed in the web interface, and the administrator presses the "Retrain" button to start retraining the model. The input is the evaluation results, and the output is the execution of retraining or the adoption of the model.

[0426] Step 9:

[0427] The server generates high-performance talk scripts based on the trained model and distributes them to all workers. Specifically, it inputs a prompt statement such as "Generate a new customer service talk script" into the model and retrieves the generated script. The input is the prompt statement, and the output is the generated talk script.

[0428] Step 10:

[0429] The terminal uses an emotion recognition engine to recognize the user's emotions in real time while serving the customer, and sends that information to the server. The server then dynamically adjusts the generated conversation script based on this information. For example, the terminal can input a prompt such as, "If the user is dissatisfied, generate a script with a more polite and comforting tone." The input is real-time emotional information, and the output is the adjusted conversation script.

[0430] Step 11:

[0431] After a training session, the user (worker) records their feedback through a dedicated interface. Specifically, after the training session, they press the "Feedback" button and enter their opinions and thoughts in the comments field. The input is the worker's feedback, and the output is the feedback data recorded on the server.

[0432] Step 12:

[0433] The server analyzes the collected feedback and retrains the model as needed. Specifically, it inputs the feedback data into an analysis script, identifies areas for improvement, and then retrains the model based on this. The input is the feedback data, and the output is an improved model.

[0434] This process allows the system to generate high-quality customer service scripts and provide all staff with a uniformly high level of customer service skills. The introduction of an emotion engine also enables personalized responses based on the user's real-time emotions.

[0435] (Application example 2)

[0436] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0437] In conventional customer service work, the skills and know-how of highly productive workers are individual and cannot be easily transferred to other workers. This makes it difficult for all workers to have consistently high levels of customer service skills, leading to variations in customer satisfaction. Furthermore, it is not possible to respond appropriately to customers' emotions, which can result in a decline in the quality of service.

[0438] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0439] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for analyzing the voices of the workers in real time and recognizing customer emotions, means for dynamically presenting optimal talk scripts based on the customer emotions, and means for collecting feedback from the workers and further improving the AI ​​model. This allows all workers to have uniformly high levels of customer service skills and be able to respond appropriately to customers' emotions, which is expected to improve customer satisfaction.

[0440] A "highly productive worker" is a worker who has high productivity and skills in customer service work and can increase customer satisfaction.

[0441] A "talk script" refers to a series of responses and greetings that a worker uses when serving a customer.

[0442] "Know-how" refers to specific practical methods based on the skills, knowledge, and experience of highly productive workers.

[0443] "Means of recording" refers to a method in which highly productive workers record conversations and know-how while serving customers on a terminal and save it as data that can be analyzed later.

[0444] "Preprocessing means" refers to the technical process of converting recorded data into a format that is easier to analyze and removing noise and unnecessary information.

[0445] "Generative AI" is a form of AI that learns from large amounts of data, recognizes patterns, and generates new information and responses.

[0446] "Training" is the process of feeding preprocessed data into a generative artificial intelligence model and training it based on human supervision.

[0447] "Means for evaluating and improving performance" refers to methods for evaluating whether a generative artificial intelligence model is functioning properly and providing feedback and re-training to improve performance as necessary.

[0448] The "distribution means" is a method for providing the generated high-performance talk script to all workers so that it can be used uniformly.

[0449] "Means for analyzing voice in real time" refers to technology that analyzes the voice of workers on the spot and instantly understands the content of conversations with customers.

[0450] "Means for recognizing emotions" refers to technology that analyzes and identifies a customer's emotional state from real-time voice data and facial expressions.

[0451] The "dynamic presentation means" is a method of instantly changing an appropriate talk script according to the recognized customer emotions and providing it to the worker.

[0452] "Means for collecting feedback" refers to technology that records evaluations and opinions from workers after use and uses them to improve generative artificial intelligence.

[0453] The system embodying the present invention aims to standardize the customer service skills of highly productive workers so that all workers can provide high-level customer service. This system is realized by combining multiple means.

[0454] First, the user records the conversations and interactions they have with customers using a smartphone or smart glasses, and the recorded data is uploaded to a server.

[0455] The server then preprocesses the uploaded data: the audio data is converted to text using Google Cloud Speech-to-Text, noise and unnecessary information are removed, and the text data is formatted into a unified format.

[0456] After preprocessing, the data is trained by a generative artificial intelligence model (e.g., OpenAI GPT-4). The server then uses this data to execute a learning process to generate highly productive worker skills and effective conversation scripts. This learning process also takes into account customer emotions, enabling a more human-like response.

[0457] The generated generative artificial intelligence model is then evaluated for its performance. The server evaluates the model's performance using a separate validation dataset and improves the model as necessary based on the results.

[0458] After the model has been fully optimized, this high-performance conversation script is distributed to the worker. The worker then uses the distributed script to serve customers. During this process, the worker's voice is analyzed in real time, and the customer's emotions are recognized using the Microsoft Azure Emotion API. Based on this information, the optimal conversation script is dynamically presented. For example, if a customer expresses dissatisfaction, the script displayed reads, "I'm sorry. I'd like to know more about this. What is the problem?"

[0459] After the training session, the workers record their feedback in a dedicated interface and send it to the server, which analyzes the collected feedback and retrains the generative AI model as needed to improve its quality.

[0460] These processes will ensure that all staff have a uniformly high level of customer service skills, enabling them to respond appropriately to customers' emotions, and are expected to improve customer satisfaction.

[0461] Specific examples

[0462] Prompt sentence for generative AI model

[0463] "Generate a conversation script to be used when a customer is unhappy. For example, include phrases such as, 'Sorry, customer. What is the problem?'"

[0464] In this way, with the right prompts, the generative artificial intelligence model can generate high-quality conversation scripts, ensuring that all workers deliver a consistent, high-quality experience to customers.

[0465] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0466] Step 1:

[0467] Data collection

[0468] Input: Highly productive workers use smartphones or smart glasses to record conversations and responses while serving customers.

[0469] Processing: The device temporarily stores the recorded data.

[0470] Output: Saved audio file.

[0471] Specific operation: The user activates the recording function of the smart device and starts recording.

[0472] Step 2:

[0473] Data upload

[0474] Input: The audio file saved in step 1.

[0475] Processing: The device uploads the audio file to the server.

[0476] Output: Audio data uploaded to the server.

[0477] Specific operation: After the user has finished recording, they use a dedicated application to send the data to the server.

[0478] Step 3:

[0479] Data Preprocessing

[0480] Input: Audio data uploaded to the server.

[0481] Processing: The server converts the audio data into text using Google Cloud Speech-to-Text, removing noise and unnecessary information.

[0482] Output: Preprocessed text data.

[0483] Specific operation: The server uses a speech recognition engine to analyze the voice data and converts the results into a standard format.

[0484] Step 4:

[0485] Learning with generative artificial intelligence models

[0486] Input: Preprocessed text data.

[0487] Processing: The server uses the OpenAI GPT-4 generative artificial intelligence model to learn useful talking scripts and know-how from the pre-processed data.

[0488] Output: A trained generative artificial intelligence model.

[0489] Specific operation: The server trains repeatedly for the specified number of epochs and generates the optimal model.

[0490] Step 5:

[0491] Model evaluation

[0492] Input: A trained generative artificial intelligence model and a separate validation dataset.

[0493] Processing: The server evaluates the performance of the generative artificial intelligence model using a validation dataset.

[0494] Output: Evaluation results and necessary improvements.

[0495] Specific operation: The server automatically runs the evaluation tool and generates performance metrics.

[0496] Step 6:

[0497] Talk script generation and distribution

[0498] Input: Trained model and evaluation results.

[0499] Processing: The server generates a high-performance talk script and distributes it to the worker.

[0500] Output: Talk script distributed to the worker.

[0501] Specific operation: The server automatically distributes the talk script via text message or application.

[0502] Step 7:

[0503] Real-time analytics and emotion recognition

[0504] Input: Real-time voice data of the worker.

[0505] Processing: The device uses the Microsoft Azure Emotion API to recognize customer emotions in real time.

[0506] Output: Emotion recognition result.

[0507] Specific operation: The device transmits voice data to the analysis server and acquires emotion data in real time.

[0508] Step 8:

[0509] Dynamic Script Presentation

[0510] Input: Emotion recognition results and talk script.

[0511] Processing: The server dynamically presents the optimal talk script to the worker based on the emotion recognition results.

[0512] Output: Optimized talk script.

[0513] Specific operation: The server analyzes the emotional data and displays an appropriate response on the worker's terminal.

[0514] Step 9:

[0515] Feedback collection

[0516] Input: Feedback from the worker.

[0517] Processing: The terminal sends the collected feedback to the server.

[0518] Output: Feedback data.

[0519] Specific operation: The user inputs feedback using a dedicated interface.

[0520] Step 10:

[0521] Retraining the Model

[0522] Input: Feedback data and an existing generative artificial intelligence model.

[0523] Processing: The server retrains the generative artificial intelligence model based on the feedback data.

[0524] Output: An improved generative artificial intelligence model.

[0525] What happens: The server analyzes the feedback and retrains the model with the new data.

[0526] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0527] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0528] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0529] [Second embodiment]

[0530] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0531] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0532] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0533] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0534] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0535] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0536] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0537] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0538] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0539] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0540] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0541] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0542] MODE FOR CARRYING OUT THE INVENTION

[0543] The present invention provides a system that collects customer service scripts and know-how from highly productive workers and trains them using generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills. An embodiment of the present invention is described in detail below.

[0544] Data collection

[0545] First, the user (high productivity worker) records voice and text data on the terminal while serving customers. The recorded voice data is the content of the conversation and response during the customer service. The user (high productivity worker) then uploads the recorded data to the server via a dedicated web interface.

[0546] Data Preprocessing

[0547] The uploaded data is preprocessed by the server, which converts the audio data into text data, removes unnecessary noise and irrelevant information, and converts different text formats into a unified format, ensuring data consistency.

[0548] Learning Process

[0549] The preprocessed data is fed to a generative artificial intelligence model. The server then executes a learning process based on the data, teaching the model the specific skills and effective speech scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized.

[0550] Model evaluation and improvement

[0551] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and, if there are any shortcomings, determines that the model needs to be retrained.

[0552] Script generation and distribution

[0553] After generating a high-performance talk script, the server distributes it to all workers. Specifically, users (workers) use the distributed talk script to conduct training sessions. In these sessions, role-playing using the new script is conducted, simulating the application of the script to real customer service work.

[0554] Gathering feedback and relearning

[0555] After the training session, the user (worker) records feedback through a dedicated interface. The server analyzes the collected feedback and performs re-training to improve the model based on the feedback. This allows the generative artificial intelligence model to be continuously improved.

[0556] Specific examples

[0557] Example 1: Collecting and learning talk scripts

[0558] 1. The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving the customer. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[0559] 2. The user uploads the recorded data to the server through a web interface.

[0560] 3. The server converts the uploaded audio data into text data and removes unnecessary noise.

[0561] 4. The preprocessed data is fed into the generative artificial intelligence, which begins learning.

[0562] Example 2: Training and feedback

[0563] 1. The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[0564] 2. The user (operator) conducts a training session using the distributed script.

[0565] 3. After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[0566] 4. The server analyzes the collected feedback and retrains the model if necessary.

[0567] This invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. Furthermore, through continuous feedback and re-learning, the generative AI model can continue to provide the latest, high-quality scripts.

[0568] The processing flow will be explained below.

[0569] Program processing flow

[0570] Step 1: Data collection

[0571] 1. User (highly productive worker)

[0572] Record audio of talk scripts used by highly productive workers in their daily customer service work.

[0573] For example: "Hello, thank you for visiting. How can I help you today?"

[0574] 2. Terminal

[0575] Stores recorded audio data and generates a corresponding text log.

[0576] Saving example: audio_dataA.wav, text_logA.txt

[0577] 3. User (highly productive worker)

[0578] The collected data is uploaded to a server via a dedicated web interface.

[0579] Upload example: Send text_logA.txt to the server.

[0580] Step 2: Data Preprocessing

[0581] 1. Server

[0582] Convert the uploaded voice data into text data (voice recognition processing).

[0583] Example of converted text: audio_dataA.wav → "Hello, and welcome to SoftBank. How can we help you today?"

[0584] 2. Server

[0585] Preprocessing is performed to standardize the converted text data into a uniform format.

[0586] Preprocessing example: Removing unnecessary symbols and noise data.

[0587] 3. Server

[0588] The preprocessed data is formatted as a talk script.

[0589] Formatting example: "Hello, welcome to SoftBank. What can I do for you today?"

[0590] Step 3: Model training

[0591] 1. Server

[0592] The formatted talk script is fed into a generative artificial intelligence model to begin the learning process.

[0593] Example learning: "Calling customers by name can lead to more effective customer service."

[0594] 2. Server

[0595] Iteratively train the data until the model is trained.

[0596] Example iteration: Run training for 100 epochs.

[0597] Step 4: Model evaluation

[0598] 1. Server

[0599] A separate validation dataset is prepared to evaluate the performance of the trained model, and validation is performed.

[0600] Validation example: Analysis of accuracy, customer satisfaction scores.

[0601] 2. User (Administrator)

[0602] Check the evaluation results of the trained model and determine if there are any deficiencies and whether the model needs to be retrained.

[0603] Example: If the assessment score is less than 95%, decide to restudy.

[0604] Step 5: Generate and distribute scripts

[0605] 1. Server

[0606] High-performance talk scripts are generated based on successful models and distributed to all workers.

[0607] Example of a distribution script: "Thank you for visiting us, Mr. / Ms. XX. What kind of service would you like today?"

[0608] 2. Terminal

[0609] Set up a training session for all workers and role-play using the new script.

[0610] Step 6: Gather feedback and improve

[0611] 1. User (operator)

[0612] Record feedback after training sessions using a dedicated interface.

[0613] Example feedback: "Customers responded very well to the new script, but often prefer a more casual tone."

[0614] 2. Server

[0615] Analyze the collected feedback, identify areas for improvement in the model, and retrain it.

[0616] Example of improvement: Use the collected feedback to further casualize the script.

[0617] Example 1

[0618] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0619] In conventional customer service operations, it was difficult to effectively share the talk scripts and know-how of highly productive workers with other workers. As a result, there was a large difference in customer service skills among individuals, resulting in an inconsistent overall service quality. Furthermore, the process of improving customer service scripts through continuous training and feedback was not carried out efficiently, making it difficult to achieve sustained operational improvement. A system was needed to solve these issues and enable all workers to acquire and maintain a high level of customer service skills.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0621] In this invention, the server includes: means for recording the talk scripts and know-how of highly productive workers; means for uploading the recorded talk scripts and know-how to the server through a dedicated interface; means for converting the uploaded voice data into text data and removing unnecessary noise and irrelevant information; means for converting the preprocessed data into a unified format; means for training a generative artificial intelligence (AI) based on the preprocessed data; means for evaluating the performance of the AI ​​model using a separate validation dataset and identifying areas for improvement; means for generating high-performance talk scripts and distributing them to all workers; means for workers to record feedback after training sessions; and means for analyzing the collected feedback and retraining the AI ​​model to continuously improve it. This enables all workers to have a uniformly high level of customer service skills, facilitating continuous improvement and improving overall service quality.

[0622] A "highly productive worker" is a worker who has outstanding skills and know-how in customer service work and demonstrates higher productivity and effectiveness than other workers.

[0623] A "talk script" is a set of predetermined conversational content and phrases used during customer service.

[0624] "Know-how" refers to practical knowledge based on specialized knowledge, skills, and experience required to effectively carry out specific tasks or operations.

[0625] A "dedicated interface" is a specific operating screen or system that allows users to input, manage, and transmit data.

[0626] A "server" is a computer system that stores and processes data via a network, and plays a central role in this system.

[0627] "Preprocessing" refers to a series of processing operations performed on raw data to prepare it in a form suitable for analysis and learning.

[0628] "Generative AI" is a type of AI that has the ability to learn patterns from large amounts of data and generate and respond to new data.

[0629] "High-performance talk scripts" refer to standard, high-quality conversation phrases and content generated by generative artificial intelligence.

[0630] "Feedback" refers to the evaluations and opinions provided by workers after training sessions or actual work performance, which are used to improve the system.

[0631] A "training session" is a training activity in which workers simulate real-world work scenarios using pre-prepared scripts.

[0632] "Retraining" is the process of retraining a generative artificial intelligence model based on new data and feedback collected to improve its performance.

[0633] "Audio data" refers to data that records the user's voice, and is used to record the contents of the conversation.

[0634] "Text data" is voice data converted into text information and is used for analysis and learning.

[0635] "Noise" refers to unnecessary information or noise that should not be included in the data being analyzed.

[0636] A "format" refers to the form or structure that allows data to be handled uniformly.

[0637] A "validation dataset" is data prepared separately from the training data and used to evaluate the performance of a generative artificial intelligence model.

[0638] MODE FOR CARRYING OUT THE INVENTION

[0639] The present invention provides a system that collects customer service scripts and know-how from highly productive workers and trains them in a generative artificial intelligence (AI) system, thereby enabling all workers to have uniformly high levels of customer service skills. The following describes an embodiment of the present invention.

[0640] Data collection

[0641] The user (highly productive worker) records voice and text data during customer service operations on a terminal. This terminal is a device with a recording function, such as a smartphone or tablet. The recorded voice data specifically includes the content of the conversation during customer service, such as "Hello, thank you for visiting our store. What can I do for you today?" The user uploads the recorded data to a server via a dedicated web interface.

[0642] Data Preprocessing

[0643] The server converts the uploaded voice data into text data using speech recognition software such as the Google Cloud Speech-to-Text API. The server then uses text analysis tools such as the NLTK library to remove unnecessary noise and irrelevant information. This ensures data consistency and converts it into a unified format (e.g., JSON).

[0644] Learning Process

[0645] The preprocessed data is fed into a generative artificial intelligence model (e.g., OpenAI's GPT-3). The server executes the learning process based on the specified number of epochs (e.g., 100 epochs). During the learning process, the model learns the specific skills and effective talking scripts of highly productive workers. This process is distributed using a high-performance GPU cluster.

[0646] Model evaluation and improvement

[0647] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. Metrics such as F1 score and accuracy are used in the evaluation. The user (administrator) can check the evaluation results through the dashboard and decide whether to retrain the model if there are any shortcomings.

[0648] Script generation and distribution

[0649] Once a high-performance talk script has been generated, the server distributes it to all workers. Specifically, the server sends notifications via email or a dedicated app. For example, a talk script such as "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?" may be distributed. Users (workers) use the distributed script to conduct training sessions and role-play.

[0650] Gathering feedback and relearning

[0651] After the training session, the user (worker) records their impressions and suggestions for improvement through a dedicated feedback interface (e.g., a web form). For example, specific opinions such as "The new script was very effective, but I thought it needed to be a little more casual in its expressions" can be recorded. The server analyzes the collected feedback using natural language processing technology and reflects it in the next re-training. This allows the generative AI model to be continuously improved.

[0652] Specific examples

[0653] Example 1: Collecting and learning talk scripts

[0654] The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving customers. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[0655] The user uploads the recorded data to a server through a web interface.

[0656] The server converts the uploaded voice data into text data and removes unnecessary noise.

[0657] The pre-processed data is fed into the generative artificial intelligence, which begins learning.

[0658] Example 2: Training and feedback

[0659] The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[0660] The user (operator) conducts a training session using the distributed script.

[0661] After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[0662] The server analyzes the collected feedback and retrains the model if necessary.

[0663] Prompt Sentence Examples

[0664] "We want to train the AI ​​based on customer service scripts written by highly productive workers. We would like you to record the following conversations and carry out the training process:

[0665] "Hello, thank you for coming. How can I help you today?"

[0666] "Thank you for visiting us, Mr. / Ms. X. What kind of service would you like today?"

[0667] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0668] Step 1: Data collection

[0669] The user (highly productive worker) records conversations during customer service duties on a device. For example, using a recording app on a smartphone, the user can record a conversation that goes something like, "Hello, thank you for visiting our store. What can I do for you today?" Once the recording is complete, the user uploads the data to a server via a dedicated web interface.

[0670] Input: Voice data during customer service

[0671] Output: Recording data is uploaded to the server

[0672] How it works: Record audio using a recording app on your smartphone and upload the file to a server using a dedicated web interface. The web interface is an application that runs on a browser.

[0673] Step 2: Data Preprocessing

[0674] The server converts the uploaded audio data into text data. It uses speech recognition software, such as the Google Cloud Speech-to-Text API, to convert the recording into text. The server then uses a text analysis tool, such as NLTK, to remove unnecessary noise and irrelevant information from the data. It also converts the text data into a standardized format, such as JSON.

[0675] Input: Uploaded audio data

[0676] Output: Preprocessed text data

[0677] What it does: The server runs speech recognition software, analyzes the resulting text data, removes noise, and converts the data into a unified format.

[0678] Step 3: The learning process

[0679] The server supplies the preprocessed text data to a generative artificial intelligence model (e.g., OpenAI's GPT-3). The learning process begins based on a specified number of epochs (e.g., 100 epochs). The model learns the specific skills and speech scripts of highly productive workers. This process is distributed using a high-performance GPU cluster.

[0680] Input: Preprocessed text data

[0681] Output: A trained generative artificial intelligence model

[0682] Specific operation: The server inputs text data into the generative AI model and performs learning for the specified number of epochs. The learning progress is recorded in the server's log.

[0683] Step 4: Model evaluation and improvement

[0684] The server evaluates the performance of the generative AI model using a separate validation dataset. The effectiveness of the model is confirmed based on evaluation indicators such as F1 score and accuracy. The user (administrator) can check the evaluation results via the dashboard and decide whether to retrain the model if any shortcomings are found.

[0685] Input: trained model, validation dataset

[0686] Output: Model performance evaluation results

[0687] Specific operation: The server inputs the validation data into the model and displays the obtained score on the dashboard. The administrator can check the results on the dashboard and instruct the model to be retrained if necessary.

[0688] Step 5: Generate and distribute scripts

[0689] The server generates high-performance conversation scripts and distributes them to all workers. Specifically, the server sends notifications via email or a dedicated app. For example, a conversation script such as "Thank you for visiting our store, Mr. / Ms. X. What kind of service would you like today?" is generated and distributed. Users (workers) use the distributed scripts to conduct training sessions and role-play.

[0690] Input: A trained model

[0691] Output: Generated talk script

[0692] How it works: The server uses the trained model to generate a talk script and sends it to the worker via email or a dedicated app. The worker receives the script and practices it during the training session.

[0693] Step 6: Gather feedback and retrain

[0694] After the training session, the user (worker) records their impressions and suggestions for improvement through a dedicated feedback interface (e.g., a web form). The server analyzes the collected feedback using natural language processing technology and continuously improves the generative AI model by retraining it.

[0695] Input: Feedback data

[0696] Output: An improved generative artificial intelligence model

[0697] Specific operation: The worker enters feedback into a web form and submits it. The server analyzes the feedback data and retrains the model as necessary. The retrained model is used to generate a new talk script.

[0698] (Application example 1)

[0699] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0700] In conventional customer service work, there was a large variation in the customer service skills and experience of individual workers, resulting in uneven customer satisfaction and work efficiency. Furthermore, it took time and effort for workers to learn new customer service scripts, making it difficult to consistently provide high-quality service. Furthermore, there was no system in place to provide effective customer service phrases in real time, which sometimes led to inappropriate responses during customer service.

[0701] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0702] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for collecting feedback from workers and further improving the AI ​​model, and means for the AI ​​to suggest customer service phrases in real time when a store clerk converses with a customer. This allows all workers to have uniformly high levels of customer service skills and enables the provision of effective customer service phrases in real time.

[0703] "Highly productive workers" refer to workers who have high productivity and excellent skills in customer service work.

[0704] A "talk script" refers to a series of phrases and responses used in customer service.

[0705] "Know-how" refers to techniques, knowledge, and methods that are effective in a specific business or task.

[0706] "Preprocessing" refers to the process of removing unnecessary noise and irrelevant information from recorded data and standardizing it into an appropriate format.

[0707] "Generative AI" refers to AI that has the ability to create new data and scripts based on input data.

[0708] "Performance evaluation of a generative artificial intelligence model" refers to the act of verifying whether the output of a generative artificial intelligence model reaches the expected level.

[0709] A "high-performance talk script" refers to a series of effective, high-quality customer service phrases optimized using generative artificial intelligence.

[0710] "Means of suggesting conversation phrases in real time" refers to a system in which generative artificial intelligence instantly suggests the most appropriate phrases when a store clerk is conversing with a customer.

[0711] The present invention is a system that collects customer service scripts and know-how from highly productive workers, trains them into a generative artificial intelligence model, and enables all workers to acquire high-level customer service skills. Specific embodiments for carrying out the invention are described below.

[0712] Data collection

[0713] First, the user (highly productive worker) records conversations during customer service using the recording function of their smartphone. The user then uploads the recorded audio data to a server via a dedicated web interface. The audio data includes the conversations and responses made during customer service.

[0714] Data Preprocessing

[0715] Once the server receives the uploaded audio data, it converts it into text using the SpeechRecognition library. It then removes unnecessary noise and irrelevant information and organizes the text data into a unified format. This preprocessing process ensures consistent data suitable for analysis.

[0716] Learning Process

[0717] The preprocessed data is fed to a generative AI model (e.g., GPT-4). The server then executes a model learning process based on this data, teaching the model the specific skills and effective speech scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized.

[0718] Model evaluation and improvement

[0719] After the training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model, and the user (administrator) can review the evaluation results and determine whether the model needs to be retrained if there are any shortcomings.

[0720] Script generation and distribution

[0721] After generating a high-performance talk script, the server distributes it to all workers. The users (workers) then use the distributed talk script to conduct training sessions. In these sessions, role-playing using the new script is conducted, simulating the application of the script to real customer service tasks.

[0722] Real-time phrase suggestions

[0723] During customer service, the server uses generative artificial intelligence to suggest optimal response phrases in real time. The user (worker) receives these in real time on their smartphone and uses them in conversation with the customer. This enables high-quality responses that are immediately suited to the situation.

[0724] Gathering feedback and relearning

[0725] After the training session, the user (worker) records feedback using a dedicated interface, and the server analyzes this feedback and performs re-training to further improve the generative artificial intelligence model.

[0726] Specific examples

[0727] For example, imagine a scenario where a new sales associate greets a customer with, "Hello, thank you for your inquiry. How can I help you today?" This conversation is recorded on a smartphone and uploaded through an application. The generative AI model analyzes this data and suggests an optimized script.

[0728] Example prompts for generative AI models

[0729] An example prompt is:

[0730] Prompt: Please optimize the following customer service script.

[0731] input:

[0732] "Hello, thank you for coming. How can I help you today?"

[0733] the goal:

[0734] 1. Address customers in a bright, friendly tone.

[0735] 2. Include questions to quickly understand the customer's needs.

[0736] Example output:

[0737] "Hello! Welcome. How can I help you today?"

[0738] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0739] Step 1:

[0740] The user (highly productive worker) records conversations while serving customers using their smartphone.

[0741] Input: Recording of voice conversations using a smartphone

[0742] Output: Recorded audio data file (e.g. .wav)

[0743] How it works: Use a dedicated application on your smartphone to record audio data through the microphone. Once the recording is complete, press the save button to create a data file.

[0744] Step 2:

[0745] Users upload the recorded audio data to the server via a dedicated web interface.

[0746] Input: Recorded audio data file

[0747] Output: Audio data file stored on the server

[0748] Specific operations: Access the web interface, press the upload button to select a file, and press the send button to transfer the audio data to the server.

[0749] Step 3:

[0750] The server receives the uploaded voice data and converts it into text data using the SpeechRecognition library.

[0751] Input: Uploaded audio data file

[0752] Output: Text data converted from audio data

[0753] Specific operation: Analyzes the audio file using the SpeechRecognition library and converts the audio signal into a string. The converted text data is temporarily saved.

[0754] Step 4:

[0755] The server preprocesses the text data, removing unnecessary noise and irrelevant information, and arranging it into a unified format.

[0756] Input: Converted text data

[0757] Output: Preprocessed clean text data

[0758] Specific operation: Using noise filtering algorithms and regular expressions, unnecessary parts are removed from text data and it is formatted into a unified format.

[0759] Step 5:

[0760] The pre-processed data is fed into a generative artificial intelligence model to begin the learning process.

[0761] Input: Preprocessed text data

[0762] Output: Trained generative AI model (initial version)

[0763] What it does: It feeds data to a generative AI model (e.g., GPT-4) and trains it for a specified number of epochs. The model learns based on a specific conversation script and response style.

[0764] Step 6:

[0765] The server evaluates the model using a separate validation dataset to evaluate the performance of the trained model.

[0766] Input: trained model and validation dataset

[0767] Output: Model performance evaluation results (accuracy, effectiveness, etc.)

[0768] What it does: It evaluates the model output using a validation dataset to measure accuracy and effectiveness, and stores the results in a database.

[0769] Step 7:

[0770] The user (administrator) checks the evaluation results and, if there are any shortcomings, instructs the model to be retrained.

[0771] Input: Model performance evaluation results

[0772] Output: Retraining instructions or model approval

[0773] Specific behavior: The administrator reviews the assessment results and determines whether retraining is necessary. If necessary, the administrator triggers the retraining process.

[0774] Step 8:

[0775] After the high-performance talk script is generated, the server distributes it to all workers.

[0776] Input: The final trained generative artificial intelligence model

[0777] Output: Distributed high performance talk script

[0778] Specific operation: The server sends the script to each worker's terminal and instructs them to conduct a training session using the new script.

[0779] Step 9:

[0780] After the training session, the user (worker) records feedback using a dedicated interface.

[0781] Input: Operator feedback during training session

[0782] Output: Recorded feedback data

[0783] What it does: Workers complete a training session and then use a dedicated web interface to provide feedback on the effectiveness of the script.

[0784] Step 10:

[0785] The server analyzes the collected feedback and performs re-training to further improve the generative artificial intelligence model.

[0786] Input: Recorded feedback data

[0787] Output: An improved generative artificial intelligence model

[0788] What it does: Update the generative AI model based on the feedback data, then run another epoch of training, reassessing the model's accuracy and effectiveness, and continuously optimizing it.

[0789] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0790] MODE FOR CARRYING OUT THE INVENTION

[0791] The present invention provides a system that collects the customer service scripts and know-how of highly productive workers and trains them into a generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills, and also combines it with an emotion engine that recognizes the emotions of users.Embodiments of the present invention are described in detail below.

[0792] Data collection

[0793] First, the user (high productivity worker) records voice and text data on the terminal while serving customers. The recorded voice data is the content of the conversation and response during the customer service. The user (high productivity worker) then uploads the recorded data to the server via a dedicated web interface.

[0794] Data Preprocessing

[0795] The uploaded data is preprocessed by the server, which converts the audio data into text data, removes unnecessary noise and irrelevant information, and converts different text formats into a unified format, ensuring data consistency.

[0796] emotion recognition

[0797] The acquired voice data may contain the user's emotional information. The terminal uses an emotion engine to recognize the user's emotion from the data. The emotion recognition result is used in subsequent processing.

[0798] Learning Process

[0799] The preprocessed data is supplied to the generative AI model. The server then executes a learning process based on this data, teaching the model the specific skills and effective conversation scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is fully optimized. The emotional information recognized by the emotion engine can also be taught to the generative AI model, enabling more human-like responses.

[0800] Model evaluation and improvement

[0801] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and, if there are any shortcomings, determines that the model needs to be retrained.

[0802] Script generation and dynamic adjustment

[0803] Once a high-performance talk script is generated, it is distributed to all workers. The talk script is dynamically adjusted according to the user's emotions. Specifically, the server recognizes the user's emotions in real time using an emotion engine and adjusts the talk script based on that information. For example, if the user is dissatisfied, a script with a more polite and comforting tone is used.

[0804] Gathering feedback and relearning

[0805] After the training session, the user (worker) records feedback through a dedicated interface. The server analyzes the collected feedback and performs re-training to improve the model based on the feedback. This allows the generative artificial intelligence model to be continuously improved.

[0806] Specific examples

[0807] Example 1: Collecting and learning talk scripts

[0808] 1. The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving the customer. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[0809] 2. The user uploads the recorded data to the server through a web interface.

[0810] 3. The server converts the uploaded audio data into text data and removes unnecessary noise.

[0811] 4. The preprocessed data is fed to the generative AI, where learning begins. The emotion engine also extracts the user's emotional information from the voice data, which is then used as learning data.

[0812] Example 2: Dynamic talk script generation and feedback

[0813] 1. The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[0814] 2. The user (worker) conducts a training session using the distributed script. During the training, the emotion engine recognizes the user's emotions in real time and adjusts the script accordingly.

[0815] 3. After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[0816] 4. The server analyzes the collected feedback and retrains the model if necessary.

[0817] In this way, the system of the present invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. In addition, the introduction of an emotion engine enables more personalized customer service, which will also improve customer satisfaction.

[0818] The processing flow will be explained below.

[0819] Program processing flow

[0820] Step 1: Data collection

[0821] 1. User (highly productive worker)

[0822] Record audio of talk scripts used by highly productive workers in their daily customer service work.

[0823] For example: "Hello, thank you for visiting. How can I help you today?"

[0824] 2. Terminal

[0825] Stores recorded audio data and generates a corresponding text log.

[0826] Saving example: audio_dataA.wav, text_logA.txt

[0827] 3. User (highly productive worker)

[0828] The collected data is uploaded to a server via a dedicated web interface.

[0829] Upload example: Send text_logA.txt to the server.

[0830] Step 2: Data Preprocessing

[0831] 1. Server

[0832] Convert the uploaded voice data into text data (voice recognition processing).

[0833] Example of converted text: audio_dataA.wav → "Hello, thank you for visiting. How can we help you today?"

[0834] 2. Server

[0835] Preprocessing is performed to standardize the converted text data into a uniform format.

[0836] Preprocessing example: Removing unnecessary symbols and noise data.

[0837] 3. Server

[0838] The preprocessed data is formatted as a talk script.

[0839] Formatting example: "Hello, thank you for visiting. What are you looking for today?"

[0840] Step 3: Emotion Recognition

[0841] 1. Terminal

[0842] The acquired voice data is analyzed by an emotion engine to identify the user's emotions.

[0843] Analysis example: Identifying emotions such as "joy," "anger," "sadness," and "surprise" from voice.

[0844] 2. Terminal

[0845] The identified emotion information is transmitted to a server together with the text data.

[0846] Example of transmission: Emotion information "joy" is sent to the server.

[0847] Step 4: Model training

[0848] 1. Server

[0849] The formatted talk script and the emotional information obtained from the emotion engine are fed into the generative artificial intelligence model to begin the learning process.

[0850] Example learning: "Calling customers by name can lead to more effective customer service."

[0851] 2. Server

[0852] Iteratively train the data until the model is trained.

[0853] Example iteration: Run training for 100 epochs.

[0854] Step 5: Model evaluation

[0855] 1. Server

[0856] A separate validation dataset is prepared to evaluate the performance of the trained model, and validation is performed.

[0857] Validation example: Analysis of accuracy, customer satisfaction scores.

[0858] 2. User (Administrator)

[0859] Check the evaluation results of the trained model and determine if there are any deficiencies and whether the model needs to be retrained.

[0860] Example: If the assessment score is less than 95%, decide to restudy.

[0861] Step 6: Generate and distribute scripts

[0862] 1. Server

[0863] High-performance talk scripts are generated based on successful models and distributed to all workers.

[0864] Example of a distribution script: "Thank you for visiting us, Mr. / Ms. XX. What kind of service would you like today?"

[0865] 2. Terminal

[0866] Set up a training session for all workers and role-play using the new script.

[0867] Step 7: Dynamic Adjustment

[0868] 1. Terminal

[0869] During a training session, an emotion engine is used in real time to recognize the user's emotions.

[0870] Analysis example: If the user feels "unhappy" during training, a corresponding script is recommended.

[0871] 2. Server

[0872] The talk script is dynamically adjusted based on emotional information obtained in real time.

[0873] Example of adjustment: If the user is perceived as "unhappy," change the talk script to a more polite and comforting tone.

[0874] Step 8: Gather feedback and improve

[0875] 1. User (operator)

[0876] Record feedback after training sessions using a dedicated interface.

[0877] Example feedback: "Customers responded very well to the new script, but often prefer a more casual tone."

[0878] 2. Server

[0879] Analyze the collected feedback, identify areas for improvement in the model, and retrain it.

[0880] Example of improvement: Use the collected feedback to further casualize the script.

[0881] In this way, the system of the present invention standardizes the quality of customer service scripts and improves overall productivity. Furthermore, the introduction of an emotion engine enables more personalized customer service, thereby improving customer satisfaction.

[0882] Example 2

[0883] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0884] It is necessary to uniformly impart the skills and know-how of highly productive workers to all workers, so that they can have a high level of customer service skills. There is also a need to improve customer satisfaction by recognizing users' emotions in real time and providing personalized responses. As it is difficult to effectively solve these issues using conventional methods, a new system is needed.

[0885] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0886] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for collecting feedback from workers and further improving the AI ​​model, and means for recognizing user emotions and dynamically adjusting the talk scripts based on that information. This enables all workers to have uniformly high levels of customer service skills and realizes personalized responses according to the user's emotions.

[0887] A "highly productive worker" refers to a worker who has excellent customer service skills and efficiency, and who achieves excellent results in their work.

[0888] A "talk script" refers to a document or data that compiles standard conversation flows and phrases used when serving or interacting with customers.

[0889] "Know-how" refers to practical knowledge and skills for performing specific tasks or operations.

[0890] "Preprocessing" refers to the process of removing noise and standardizing the format of data before it is fed into a generative artificial intelligence model.

[0891] "Generative AI" refers to AI that generates new data and responses based on learned data.

[0892] A "generative artificial intelligence model" refers to an algorithm or structure for prediction or generation that is constructed based on the learning results of generative artificial intelligence.

[0893] "Evaluating and improving performance" refers to verifying the accuracy and effectiveness of the generative AI model, and retraining or adjusting the model as necessary.

[0894] "High-performance talk scripts" refer to talk scripts generated by a generative artificial intelligence model that are evaluated as having high customer service skills and effectiveness.

[0895] "Feedback" refers to collecting opinions and impressions that workers gain through training sessions and on-the-job tasks.

[0896] "Recognizing user emotions" refers to analyzing and acquiring the user's emotional state from voice and text data.

[0897] "Dynamic adjustment" refers to changing the content of the talk script provided in real time according to the user's situation and emotions.

[0898] MODE FOR CARRYING OUT THE INVENTION

[0899] The present invention provides a system that collects the customer service scripts and know-how of highly productive workers and trains them into a generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills, and also combines it with an emotion engine that recognizes the emotions of users.Embodiments of the present invention are described in detail below.

[0900] Data collection

[0901] First, the user (highly productive worker) records voice and text data on the device while serving customers. Specifically, the device's recording function is used to record the conversation. For example, a script such as "Hello, thank you for visiting. What can I do for you today?" is recorded. The recorded voice data and related text data are then uploaded to the server via a dedicated web interface. Using a web browser, the user authenticates on the login screen, clicks the "File Upload" button, selects the data, and then clicks the "Send" button to complete the upload.

[0902] Data Preprocessing

[0903] The uploaded data is preprocessed by the server. First, speech recognition software (e.g., Google Cloud Speech-to-Text API) is used to convert the audio data into text data. The server sends the audio file to the speech recognition API and receives the converted text data. Then, it removes unnecessary noise and irrelevant information from the text data and standardizes the data into a standardized format using natural language processing tools (e.g., Python's NLTK library). This generates data in a standardized format.

[0904] emotion recognition

[0905] The acquired voice data may contain the user's emotional information. The terminal uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotion from this data. Specifically, the terminal inputs the voice data into the emotion recognition engine and obtains the output emotion label. This allows the emotional information to be extracted.

[0906] Learning Process

[0907] The preprocessed data is fed into a generative AI model (for example, OpenAI's GPT model). The server then runs a learning process based on this data, teaching the model the specific skills and effective conversation scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized. The emotional information recognized by the emotion engine can also be taught to the generative AI model, enabling it to respond more like a human.

[0908] Model evaluation and improvement

[0909] After training is complete, the server evaluates the performance of the generative artificial intelligence model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and determines whether the model needs to be retrained if there are any shortcomings. The evaluation results are displayed visually in a web interface, and the administrator can start retraining the model by pressing the "Retrain" button.

[0910] Script generation and dynamic adjustment

[0911] Once a high-performance talk script has been generated, it is distributed to all workers. The talk script is dynamically adjusted according to the user's emotions. Specifically, the server recognizes the user's emotions in real time using an emotion recognition engine and adjusts the talk script based on that information. For example, a prompt such as "For dissatisfied users, generate a script with a more polite and comforting tone" can be input and the generated script can be obtained.

[0912] Gathering feedback and relearning

[0913] After the training session, the user (worker) records feedback through a dedicated interface. For example, "The new script was very effective, but I thought it needed to be a little more casual." The server analyzes the collected feedback and retrains the model as needed. This allows the generative artificial intelligence model to continuously improve.

[0914] In this way, the system of the present invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. Furthermore, the introduction of an emotion engine enables more personalized customer service, which will also improve customer satisfaction.

[0915] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0916] Step 1:

[0917] The user records the conversation during the customer service session using the device's recording function. Specifically, the user presses the recording button on the device to start the conversation, and presses the button again to stop recording when finished. The input is audio data, and the output is an audio file saved on the device.

[0918] Step 2:

[0919] The user uploads the recorded audio data and associated text data to the server through a dedicated web interface. Specifically, the user opens a web browser, authenticates, clicks the "File Upload" button, selects the audio file, and clicks the "Send" button. The input is the audio file and text data, and the output is the data uploaded to the server.

[0920] Step 3:

[0921] The server converts the uploaded voice data into text data using voice recognition software (e.g., Google Cloud Speech-to-Text API). Specifically, the server sends the voice file to the API and receives the converted text data. The input is voice data, and the output is text data.

[0922] Step 4:

[0923] The server uses natural language processing tools (e.g., Python's NLTK library) to remove unnecessary noise from the converted text data and standardize the data into a format. Specifically, it analyzes the text data, removes unnecessary words and sentences, and converts them into grammatically correct sentences. The input is text data, and the output is formatted text data.

[0924] Step 5:

[0925] The device uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotion from the recorded voice data. Specifically, the voice data is input to the engine, and an emotion label is output. The input is the voice data, and the output is emotion information.

[0926] Step 6:

[0927] The server feeds the preprocessed text data and sentiment information to a generative artificial intelligence model (e.g., OpenAI's GPT model). The server inputs the dataset into the model's training script and runs training for a specified number of epochs. The input is the preprocessed data, and the output is a trained model.

[0928] Step 7:

[0929] The server uses a separate validation dataset to evaluate the performance of the trained model. Specifically, it inputs test data to the model and runs a script that calculates evaluation metrics such as precision and recall. The input is the validation dataset, and the output is the evaluation results.

[0930] Step 8:

[0931] The user (administrator) checks the evaluation results and decides whether to retrain if necessary. The evaluation results are visually displayed in the web interface, and the administrator presses the "Retrain" button to start retraining the model. The input is the evaluation results, and the output is the execution of retraining or the adoption of the model.

[0932] Step 9:

[0933] The server generates high-performance talk scripts based on the trained model and distributes them to all workers. Specifically, it inputs a prompt statement such as "Generate a new customer service talk script" into the model and retrieves the generated script. The input is the prompt statement, and the output is the generated talk script.

[0934] Step 10:

[0935] The terminal uses an emotion recognition engine to recognize the user's emotions in real time while serving the customer, and sends that information to the server. The server then dynamically adjusts the generated conversation script based on this information. For example, the terminal can input a prompt such as, "If the user is dissatisfied, generate a script with a more polite and comforting tone." The input is real-time emotional information, and the output is the adjusted conversation script.

[0936] Step 11:

[0937] After a training session, the user (worker) records their feedback through a dedicated interface. Specifically, after the training session, they press the "Feedback" button and enter their opinions and thoughts in the comments field. The input is the worker's feedback, and the output is the feedback data recorded on the server.

[0938] Step 12:

[0939] The server analyzes the collected feedback and retrains the model as needed. Specifically, it inputs the feedback data into an analysis script, identifies areas for improvement, and then retrains the model based on this. The input is the feedback data, and the output is an improved model.

[0940] This process allows the system to generate high-quality customer service scripts and provide all staff with a uniformly high level of customer service skills. The introduction of an emotion engine also enables personalized responses based on the user's real-time emotions.

[0941] (Application example 2)

[0942] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0943] In conventional customer service work, the skills and know-how of highly productive workers are individual and cannot be easily transferred to other workers. This makes it difficult for all workers to have consistently high levels of customer service skills, leading to variations in customer satisfaction. Furthermore, it is not possible to respond appropriately to customers' emotions, which can result in a decline in the quality of service.

[0944] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0945] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for analyzing the voices of the workers in real time and recognizing customer emotions, means for dynamically presenting optimal talk scripts based on the customer emotions, and means for collecting feedback from the workers and further improving the AI ​​model. This allows all workers to have uniformly high levels of customer service skills and be able to respond appropriately to customers' emotions, which is expected to improve customer satisfaction.

[0946] A "highly productive worker" is a worker who has high productivity and skills in customer service work and can increase customer satisfaction.

[0947] A "talk script" refers to a series of responses and greetings that a worker uses when serving a customer.

[0948] "Know-how" refers to specific practical methods based on the skills, knowledge, and experience of highly productive workers.

[0949] "Means of recording" refers to a method in which highly productive workers record conversations and know-how while serving customers on a terminal and save it as data that can be analyzed later.

[0950] "Preprocessing means" refers to the technical process of converting recorded data into a format that is easier to analyze and removing noise and unnecessary information.

[0951] "Generative AI" is a form of AI that learns from large amounts of data, recognizes patterns, and generates new information and responses.

[0952] "Training" is the process of feeding preprocessed data into a generative artificial intelligence model and training it based on human supervision.

[0953] "Means for evaluating and improving performance" refers to methods for evaluating whether a generative artificial intelligence model is functioning properly and providing feedback and re-training to improve performance as necessary.

[0954] The "distribution means" is a method for providing the generated high-performance talk script to all workers so that it can be used uniformly.

[0955] "Means for analyzing voice in real time" refers to technology that analyzes the voice of workers on the spot and instantly understands the content of conversations with customers.

[0956] "Means for recognizing emotions" refers to technology that analyzes and identifies a customer's emotional state from real-time voice data and facial expressions.

[0957] The "dynamic presentation means" is a method of instantly changing an appropriate talk script according to the recognized customer emotions and providing it to the worker.

[0958] "Means for collecting feedback" refers to technology that records evaluations and opinions from workers after use and uses them to improve generative artificial intelligence.

[0959] The system embodying the present invention aims to standardize the customer service skills of highly productive workers so that all workers can provide high-level customer service. This system is realized by combining multiple means.

[0960] First, the user records the conversations and interactions they have with customers using a smartphone or smart glasses, and the recorded data is uploaded to a server.

[0961] The server then preprocesses the uploaded data: the audio data is converted to text using Google Cloud Speech-to-Text, noise and unnecessary information are removed, and the text data is formatted into a unified format.

[0962] After preprocessing, the data is trained by a generative artificial intelligence model (e.g., OpenAI GPT-4). The server then uses this data to execute a learning process to generate highly productive worker skills and effective conversation scripts. This learning process also takes into account customer emotions, enabling a more human-like response.

[0963] The generated generative artificial intelligence model is then evaluated for its performance. The server evaluates the model's performance using a separate validation dataset and improves the model as necessary based on the results.

[0964] After the model has been fully optimized, this high-performance conversation script is distributed to the worker. The worker then uses the distributed script to serve customers. During this process, the worker's voice is analyzed in real time, and the customer's emotions are recognized using the Microsoft Azure Emotion API. Based on this information, the optimal conversation script is dynamically presented. For example, if a customer expresses dissatisfaction, the script displayed reads, "I'm sorry. I'd like to know more about this. What is the problem?"

[0965] After the training session, the workers record their feedback in a dedicated interface and send it to the server, which analyzes the collected feedback and retrains the generative AI model as needed to improve its quality.

[0966] These processes will ensure that all staff have a uniformly high level of customer service skills, enabling them to respond appropriately to customers' emotions, and are expected to improve customer satisfaction.

[0967] Specific examples

[0968] Prompt sentence for generative AI model

[0969] "Generate a conversation script to be used when a customer is unhappy. For example, include phrases such as, 'Sorry, customer. What is the problem?'"

[0970] In this way, with the right prompts, the generative artificial intelligence model can generate high-quality conversation scripts, ensuring that all workers deliver a consistent, high-quality experience to customers.

[0971] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0972] Step 1:

[0973] Data collection

[0974] Input: Highly productive workers use smartphones or smart glasses to record conversations and responses while serving customers.

[0975] Processing: The device temporarily stores the recorded data.

[0976] Output: Saved audio file.

[0977] Specific operation: The user activates the recording function of the smart device and starts recording.

[0978] Step 2:

[0979] Data upload

[0980] Input: The audio file saved in step 1.

[0981] Processing: The device uploads the audio file to the server.

[0982] Output: Audio data uploaded to the server.

[0983] Specific operation: After the user has finished recording, they use a dedicated application to send the data to the server.

[0984] Step 3:

[0985] Data Preprocessing

[0986] Input: Audio data uploaded to the server.

[0987] Processing: The server converts the audio data into text using Google Cloud Speech-to-Text, removing noise and unnecessary information.

[0988] Output: Preprocessed text data.

[0989] Specific operation: The server uses a speech recognition engine to analyze the voice data and converts the results into a standard format.

[0990] Step 4:

[0991] Learning with generative artificial intelligence models

[0992] Input: Preprocessed text data.

[0993] Processing: The server uses the OpenAI GPT-4 generative artificial intelligence model to learn useful talking scripts and know-how from the pre-processed data.

[0994] Output: A trained generative artificial intelligence model.

[0995] Specific operation: The server trains repeatedly for the specified number of epochs and generates the optimal model.

[0996] Step 5:

[0997] Model evaluation

[0998] Input: A trained generative artificial intelligence model and a separate validation dataset.

[0999] Processing: The server evaluates the performance of the generative artificial intelligence model using a validation dataset.

[1000] Output: Evaluation results and necessary improvements.

[1001] Specific operation: The server automatically runs the evaluation tool and generates performance metrics.

[1002] Step 6:

[1003] Talk script generation and distribution

[1004] Input: Trained model and evaluation results.

[1005] Processing: The server generates a high-performance talk script and distributes it to the worker.

[1006] Output: Talk script distributed to the worker.

[1007] Specific operation: The server automatically distributes the talk script via text message or application.

[1008] Step 7:

[1009] Real-time analytics and emotion recognition

[1010] Input: Real-time voice data of the worker.

[1011] Processing: The device uses the Microsoft Azure Emotion API to recognize customer emotions in real time.

[1012] Output: Emotion recognition result.

[1013] Specific operation: The device transmits voice data to the analysis server and acquires emotion data in real time.

[1014] Step 8:

[1015] Dynamic Script Presentation

[1016] Input: Emotion recognition results and talk script.

[1017] Processing: The server dynamically presents the optimal talk script to the worker based on the emotion recognition results.

[1018] Output: Optimized talk script.

[1019] Specific operation: The server analyzes the emotional data and displays an appropriate response on the worker's terminal.

[1020] Step 9:

[1021] Feedback collection

[1022] Input: Feedback from the worker.

[1023] Processing: The terminal sends the collected feedback to the server.

[1024] Output: Feedback data.

[1025] Specific operation: The user inputs feedback using a dedicated interface.

[1026] Step 10:

[1027] Retraining the Model

[1028] Input: Feedback data and an existing generative artificial intelligence model.

[1029] Processing: The server retrains the generative artificial intelligence model based on the feedback data.

[1030] Output: An improved generative artificial intelligence model.

[1031] What happens: The server analyzes the feedback and retrains the model with the new data.

[1032] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1033] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1034] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1035] [Third embodiment]

[1036] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1037] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1038] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1039] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1040] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1041] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1042] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1043] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1044] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1045] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1046] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1047] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1048] MODE FOR CARRYING OUT THE INVENTION

[1049] The present invention provides a system that collects customer service scripts and know-how from highly productive workers and trains them using generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills. An embodiment of the present invention is described in detail below.

[1050] Data collection

[1051] First, the user (high productivity worker) records voice and text data on the terminal while serving customers. The recorded voice data is the content of the conversation and response during the customer service. The user (high productivity worker) then uploads the recorded data to the server via a dedicated web interface.

[1052] Data Preprocessing

[1053] The uploaded data is preprocessed by the server, which converts the audio data into text data, removes unnecessary noise and irrelevant information, and converts different text formats into a unified format, ensuring data consistency.

[1054] Learning Process

[1055] The preprocessed data is fed to a generative artificial intelligence model. The server then executes a learning process based on the data, teaching the model the specific skills and effective speech scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized.

[1056] Model evaluation and improvement

[1057] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and, if there are any shortcomings, determines that the model needs to be retrained.

[1058] Script generation and distribution

[1059] After generating a high-performance talk script, the server distributes it to all workers. Specifically, users (workers) use the distributed talk script to conduct training sessions. In these sessions, role-playing using the new script is conducted, simulating the application of the script to real customer service work.

[1060] Gathering feedback and relearning

[1061] After the training session, the user (worker) records feedback through a dedicated interface. The server analyzes the collected feedback and performs re-training to improve the model based on the feedback. This allows the generative artificial intelligence model to be continuously improved.

[1062] Specific examples

[1063] Example 1: Collecting and learning talk scripts

[1064] 1. The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving the customer. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[1065] 2. The user uploads the recorded data to the server through a web interface.

[1066] 3. The server converts the uploaded audio data into text data and removes unnecessary noise.

[1067] 4. The preprocessed data is fed into the generative artificial intelligence, which begins learning.

[1068] Example 2: Training and feedback

[1069] 1. The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[1070] 2. The user (operator) conducts a training session using the distributed script.

[1071] 3. After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[1072] 4. The server analyzes the collected feedback and retrains the model if necessary.

[1073] This invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. Furthermore, through continuous feedback and re-learning, the generative AI model can continue to provide the latest, high-quality scripts.

[1074] The processing flow will be explained below.

[1075] Program processing flow

[1076] Step 1: Data collection

[1077] 1. User (highly productive worker)

[1078] Record audio of talk scripts used by highly productive workers in their daily customer service work.

[1079] For example: "Hello, thank you for visiting. How can I help you today?"

[1080] 2. Terminal

[1081] Stores recorded audio data and generates a corresponding text log.

[1082] Saving example: audio_dataA.wav, text_logA.txt

[1083] 3. User (highly productive worker)

[1084] The collected data is uploaded to a server via a dedicated web interface.

[1085] Upload example: Send text_logA.txt to the server.

[1086] Step 2: Data Preprocessing

[1087] 1. Server

[1088] Convert the uploaded voice data into text data (voice recognition processing).

[1089] Example of converted text: audio_dataA.wav → "Hello, and welcome to SoftBank. How can we help you today?"

[1090] 2. Server

[1091] Preprocessing is performed to standardize the converted text data into a uniform format.

[1092] Preprocessing example: Removing unnecessary symbols and noise data.

[1093] 3. Server

[1094] The preprocessed data is formatted as a talk script.

[1095] Formatting example: "Hello, welcome to SoftBank. What can I do for you today?"

[1096] Step 3: Model training

[1097] 1. Server

[1098] The formatted talk script is fed into a generative artificial intelligence model to begin the learning process.

[1099] Example learning: "Calling customers by name can lead to more effective customer service."

[1100] 2. Server

[1101] Iteratively train the data until the model is trained.

[1102] Example iteration: Run training for 100 epochs.

[1103] Step 4: Model evaluation

[1104] 1. Server

[1105] A separate validation dataset is prepared to evaluate the performance of the trained model, and validation is performed.

[1106] Validation example: Analysis of accuracy, customer satisfaction scores.

[1107] 2. User (Administrator)

[1108] Check the evaluation results of the trained model and determine if there are any deficiencies and whether the model needs to be retrained.

[1109] Example: If the assessment score is less than 95%, decide to restudy.

[1110] Step 5: Generate and distribute scripts

[1111] 1. Server

[1112] High-performance talk scripts are generated based on successful models and distributed to all workers.

[1113] Example of a distribution script: "Thank you for visiting us, Mr. / Ms. XX. What kind of service would you like today?"

[1114] 2. Terminal

[1115] Set up a training session for all workers and role-play using the new script.

[1116] Step 6: Gather feedback and improve

[1117] 1. User (operator)

[1118] Record feedback after training sessions using a dedicated interface.

[1119] Example feedback: "Customers responded very well to the new script, but often prefer a more casual tone."

[1120] 2. Server

[1121] Analyze the collected feedback, identify areas for improvement in the model, and retrain it.

[1122] Example of improvement: Use the collected feedback to further casualize the script.

[1123] Example 1

[1124] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1125] In conventional customer service operations, it was difficult to effectively share the talk scripts and know-how of highly productive workers with other workers. As a result, there was a large difference in customer service skills among individuals, resulting in an inconsistent overall service quality. Furthermore, the process of improving customer service scripts through continuous training and feedback was not carried out efficiently, making it difficult to achieve sustained operational improvement. A system was needed to solve these issues and enable all workers to acquire and maintain a high level of customer service skills.

[1126] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1127] In this invention, the server includes: means for recording the talk scripts and know-how of highly productive workers; means for uploading the recorded talk scripts and know-how to the server through a dedicated interface; means for converting the uploaded voice data into text data and removing unnecessary noise and irrelevant information; means for converting the preprocessed data into a unified format; means for training a generative artificial intelligence (AI) based on the preprocessed data; means for evaluating the performance of the AI ​​model using a separate validation dataset and identifying areas for improvement; means for generating high-performance talk scripts and distributing them to all workers; means for workers to record feedback after training sessions; and means for analyzing the collected feedback and retraining the AI ​​model to continuously improve it. This enables all workers to have a uniformly high level of customer service skills, facilitating continuous improvement and improving overall service quality.

[1128] A "highly productive worker" is a worker who has outstanding skills and know-how in customer service work and demonstrates higher productivity and effectiveness than other workers.

[1129] A "talk script" is a set of predetermined conversational content and phrases used during customer service.

[1130] "Know-how" refers to practical knowledge based on specialized knowledge, skills, and experience required to effectively carry out specific tasks or operations.

[1131] A "dedicated interface" is a specific operating screen or system that allows users to input, manage, and transmit data.

[1132] A "server" is a computer system that stores and processes data via a network, and plays a central role in this system.

[1133] "Preprocessing" refers to a series of processing operations performed on raw data to prepare it in a form suitable for analysis and learning.

[1134] "Generative AI" is a type of AI that has the ability to learn patterns from large amounts of data and generate and respond to new data.

[1135] "High-performance talk scripts" refer to standard, high-quality conversation phrases and content generated by generative artificial intelligence.

[1136] "Feedback" refers to the evaluations and opinions provided by workers after training sessions or actual work performance, which are used to improve the system.

[1137] A "training session" is a training activity in which workers simulate real-world work scenarios using pre-prepared scripts.

[1138] "Retraining" is the process of retraining a generative artificial intelligence model based on new data and feedback collected to improve its performance.

[1139] "Audio data" refers to data that records the user's voice, and is used to record the contents of the conversation.

[1140] "Text data" is voice data converted into text information and is used for analysis and learning.

[1141] "Noise" refers to unnecessary information or noise that should not be included in the data being analyzed.

[1142] A "format" refers to the form or structure that allows data to be handled uniformly.

[1143] A "validation dataset" is data prepared separately from the training data and used to evaluate the performance of a generative artificial intelligence model.

[1144] MODE FOR CARRYING OUT THE INVENTION

[1145] The present invention provides a system that collects customer service scripts and know-how from highly productive workers and trains them in a generative artificial intelligence (AI) system, thereby enabling all workers to have uniformly high levels of customer service skills. The following describes an embodiment of the present invention.

[1146] Data collection

[1147] The user (highly productive worker) records voice and text data during customer service operations on a terminal. This terminal is a device with a recording function, such as a smartphone or tablet. The recorded voice data specifically includes the content of the conversation during customer service, such as "Hello, thank you for visiting our store. What can I do for you today?" The user uploads the recorded data to a server via a dedicated web interface.

[1148] Data Preprocessing

[1149] The server converts the uploaded voice data into text data using speech recognition software such as the Google Cloud Speech-to-Text API. The server then uses text analysis tools such as the NLTK library to remove unnecessary noise and irrelevant information. This ensures data consistency and converts it into a unified format (e.g., JSON).

[1150] Learning Process

[1151] The preprocessed data is fed into a generative artificial intelligence model (e.g., OpenAI's GPT-3). The server executes the learning process based on the specified number of epochs (e.g., 100 epochs). During the learning process, the model learns the specific skills and effective talking scripts of highly productive workers. This process is distributed using a high-performance GPU cluster.

[1152] Model evaluation and improvement

[1153] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. Metrics such as F1 score and accuracy are used in the evaluation. The user (administrator) can check the evaluation results through the dashboard and decide whether to retrain the model if there are any shortcomings.

[1154] Script generation and distribution

[1155] Once a high-performance talk script has been generated, the server distributes it to all workers. Specifically, the server sends notifications via email or a dedicated app. For example, a talk script such as "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?" may be distributed. Users (workers) use the distributed script to conduct training sessions and role-play.

[1156] Gathering feedback and relearning

[1157] After the training session, the user (worker) records their impressions and suggestions for improvement through a dedicated feedback interface (e.g., a web form). For example, specific opinions such as "The new script was very effective, but I thought it needed to be a little more casual in its expressions" can be recorded. The server analyzes the collected feedback using natural language processing technology and reflects it in the next re-training. This allows the generative AI model to be continuously improved.

[1158] Specific examples

[1159] Example 1: Collecting and learning talk scripts

[1160] The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving customers. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[1161] The user uploads the recorded data to a server through a web interface.

[1162] The server converts the uploaded voice data into text data and removes unnecessary noise.

[1163] The pre-processed data is fed into the generative artificial intelligence, which begins learning.

[1164] Example 2: Training and feedback

[1165] The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[1166] The user (operator) conducts a training session using the distributed script.

[1167] After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[1168] The server analyzes the collected feedback and retrains the model if necessary.

[1169] Prompt Sentence Examples

[1170] "We want to train the AI ​​based on customer service scripts written by highly productive workers. We would like you to record the following conversations and carry out the training process:

[1171] "Hello, thank you for coming. How can I help you today?"

[1172] "Thank you for visiting us, Mr. / Ms. X. What kind of service would you like today?"

[1173] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1174] Step 1: Data collection

[1175] The user (highly productive worker) records conversations during customer service duties on a device. For example, using a recording app on a smartphone, the user can record a conversation that goes something like, "Hello, thank you for visiting our store. What can I do for you today?" Once the recording is complete, the user uploads the data to a server via a dedicated web interface.

[1176] Input: Voice data during customer service

[1177] Output: Recording data is uploaded to the server

[1178] How it works: Record audio using a recording app on your smartphone and upload the file to a server using a dedicated web interface. The web interface is an application that runs on a browser.

[1179] Step 2: Data Preprocessing

[1180] The server converts the uploaded audio data into text data. It uses speech recognition software, such as the Google Cloud Speech-to-Text API, to convert the recording into text. The server then uses a text analysis tool, such as NLTK, to remove unnecessary noise and irrelevant information from the data. It also converts the text data into a standardized format, such as JSON.

[1181] Input: Uploaded audio data

[1182] Output: Preprocessed text data

[1183] What it does: The server runs speech recognition software, analyzes the resulting text data, removes noise, and converts the data into a unified format.

[1184] Step 3: The learning process

[1185] The server supplies the preprocessed text data to a generative artificial intelligence model (e.g., OpenAI's GPT-3). The learning process begins based on a specified number of epochs (e.g., 100 epochs). The model learns the specific skills and speech scripts of highly productive workers. This process is distributed using a high-performance GPU cluster.

[1186] Input: Preprocessed text data

[1187] Output: A trained generative artificial intelligence model

[1188] Specific operation: The server inputs text data into the generative AI model and performs learning for the specified number of epochs. The learning progress is recorded in the server's log.

[1189] Step 4: Model evaluation and improvement

[1190] The server evaluates the performance of the generative AI model using a separate validation dataset. The effectiveness of the model is confirmed based on evaluation indicators such as F1 score and accuracy. The user (administrator) can check the evaluation results via the dashboard and decide whether to retrain the model if any shortcomings are found.

[1191] Input: trained model, validation dataset

[1192] Output: Model performance evaluation results

[1193] Specific operation: The server inputs the validation data into the model and displays the obtained score on the dashboard. The administrator can check the results on the dashboard and instruct the model to be retrained if necessary.

[1194] Step 5: Generate and distribute scripts

[1195] The server generates high-performance conversation scripts and distributes them to all workers. Specifically, the server sends notifications via email or a dedicated app. For example, a conversation script such as "Thank you for visiting our store, Mr. / Ms. X. What kind of service would you like today?" is generated and distributed. Users (workers) use the distributed scripts to conduct training sessions and role-play.

[1196] Input: A trained model

[1197] Output: Generated talk script

[1198] How it works: The server uses the trained model to generate a talk script and sends it to the worker via email or a dedicated app. The worker receives the script and practices it during the training session.

[1199] Step 6: Gather feedback and retrain

[1200] After the training session, the user (worker) records their impressions and suggestions for improvement through a dedicated feedback interface (e.g., a web form). The server analyzes the collected feedback using natural language processing technology and continuously improves the generative AI model by retraining it.

[1201] Input: Feedback data

[1202] Output: An improved generative artificial intelligence model

[1203] Specific operation: The worker enters feedback into a web form and submits it. The server analyzes the feedback data and retrains the model as necessary. The retrained model is used to generate a new talk script.

[1204] (Application example 1)

[1205] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1206] In conventional customer service work, there was a large variation in the customer service skills and experience of individual workers, resulting in uneven customer satisfaction and work efficiency. Furthermore, it took time and effort for workers to learn new customer service scripts, making it difficult to consistently provide high-quality service. Furthermore, there was no system in place to provide effective customer service phrases in real time, which sometimes led to inappropriate responses during customer service.

[1207] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1208] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for collecting feedback from workers and further improving the AI ​​model, and means for the AI ​​to suggest customer service phrases in real time when a store clerk converses with a customer. This allows all workers to have uniformly high levels of customer service skills and enables the provision of effective customer service phrases in real time.

[1209] "Highly productive workers" refer to workers who have high productivity and excellent skills in customer service work.

[1210] A "talk script" refers to a series of phrases and responses used in customer service.

[1211] "Know-how" refers to techniques, knowledge, and methods that are effective in a specific business or task.

[1212] "Preprocessing" refers to the process of removing unnecessary noise and irrelevant information from recorded data and standardizing it into an appropriate format.

[1213] "Generative AI" refers to AI that has the ability to create new data and scripts based on input data.

[1214] "Performance evaluation of a generative artificial intelligence model" refers to the act of verifying whether the output of a generative artificial intelligence model reaches the expected level.

[1215] A "high-performance talk script" refers to a series of effective, high-quality customer service phrases optimized using generative artificial intelligence.

[1216] "Means of suggesting conversation phrases in real time" refers to a system in which generative artificial intelligence instantly suggests the most appropriate phrases when a store clerk is conversing with a customer.

[1217] The present invention is a system that collects customer service scripts and know-how from highly productive workers, trains them into a generative artificial intelligence model, and enables all workers to acquire high-level customer service skills. Specific embodiments for carrying out the invention are described below.

[1218] Data collection

[1219] First, the user (highly productive worker) records conversations during customer service using the recording function of their smartphone. The user then uploads the recorded audio data to a server via a dedicated web interface. The audio data includes the conversations and responses made during customer service.

[1220] Data Preprocessing

[1221] Once the server receives the uploaded audio data, it converts it into text using the SpeechRecognition library. It then removes unnecessary noise and irrelevant information and organizes the text data into a unified format. This preprocessing process ensures consistent data suitable for analysis.

[1222] Learning Process

[1223] The preprocessed data is fed to a generative AI model (e.g., GPT-4). The server then executes a model learning process based on this data, teaching the model the specific skills and effective speech scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized.

[1224] Model evaluation and improvement

[1225] After the training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model, and the user (administrator) can review the evaluation results and determine whether the model needs to be retrained if there are any shortcomings.

[1226] Script generation and distribution

[1227] After generating a high-performance talk script, the server distributes it to all workers. The users (workers) then use the distributed talk script to conduct training sessions. In these sessions, role-playing using the new script is conducted, simulating the application of the script to real customer service tasks.

[1228] Real-time phrase suggestions

[1229] During customer service, the server uses generative artificial intelligence to suggest optimal response phrases in real time. The user (worker) receives these in real time on their smartphone and uses them in conversation with the customer. This enables high-quality responses that are immediately suited to the situation.

[1230] Gathering feedback and relearning

[1231] After the training session, the user (worker) records feedback using a dedicated interface, and the server analyzes this feedback and performs re-training to further improve the generative artificial intelligence model.

[1232] Specific examples

[1233] For example, imagine a scenario where a new sales associate greets a customer with, "Hello, thank you for your inquiry. How can I help you today?" This conversation is recorded on a smartphone and uploaded through an application. The generative AI model analyzes this data and suggests an optimized script.

[1234] Example prompts for generative AI models

[1235] An example prompt is:

[1236] Prompt: Please optimize the following customer service script.

[1237] input:

[1238] "Hello, thank you for coming. How can I help you today?"

[1239] the goal:

[1240] 1. Address customers in a bright, friendly tone.

[1241] 2. Include questions to quickly understand the customer's needs.

[1242] Example output:

[1243] "Hello! Welcome. How can I help you today?"

[1244] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1245] Step 1:

[1246] The user (highly productive worker) records conversations while serving customers using their smartphone.

[1247] Input: Recording of voice conversations using a smartphone

[1248] Output: Recorded audio data file (e.g. .wav)

[1249] How it works: Use a dedicated application on your smartphone to record audio data through the microphone. Once the recording is complete, press the save button to create a data file.

[1250] Step 2:

[1251] Users upload the recorded audio data to the server via a dedicated web interface.

[1252] Input: Recorded audio data file

[1253] Output: Audio data file stored on the server

[1254] Specific operations: Access the web interface, press the upload button to select a file, and press the send button to transfer the audio data to the server.

[1255] Step 3:

[1256] The server receives the uploaded voice data and converts it into text data using the SpeechRecognition library.

[1257] Input: Uploaded audio data file

[1258] Output: Text data converted from audio data

[1259] Specific operation: Analyzes the audio file using the SpeechRecognition library and converts the audio signal into a string. The converted text data is temporarily saved.

[1260] Step 4:

[1261] The server preprocesses the text data, removing unnecessary noise and irrelevant information, and arranging it into a unified format.

[1262] Input: Converted text data

[1263] Output: Preprocessed clean text data

[1264] Specific operation: Using noise filtering algorithms and regular expressions, unnecessary parts are removed from text data and it is formatted into a unified format.

[1265] Step 5:

[1266] The pre-processed data is fed into a generative artificial intelligence model to begin the learning process.

[1267] Input: Preprocessed text data

[1268] Output: Trained generative AI model (initial version)

[1269] What it does: It feeds data to a generative AI model (e.g., GPT-4) and trains it for a specified number of epochs. The model learns based on a specific conversation script and response style.

[1270] Step 6:

[1271] The server evaluates the model using a separate validation dataset to evaluate the performance of the trained model.

[1272] Input: trained model and validation dataset

[1273] Output: Model performance evaluation results (accuracy, effectiveness, etc.)

[1274] What it does: It evaluates the model output using a validation dataset to measure accuracy and effectiveness, and stores the results in a database.

[1275] Step 7:

[1276] The user (administrator) checks the evaluation results and, if there are any shortcomings, instructs the model to be retrained.

[1277] Input: Model performance evaluation results

[1278] Output: Retraining instructions or model approval

[1279] Specific behavior: The administrator reviews the assessment results and determines whether retraining is necessary. If necessary, the administrator triggers the retraining process.

[1280] Step 8:

[1281] After the high-performance talk script is generated, the server distributes it to all workers.

[1282] Input: The final trained generative artificial intelligence model

[1283] Output: Distributed high performance talk script

[1284] Specific operation: The server sends the script to each worker's terminal and instructs them to conduct a training session using the new script.

[1285] Step 9:

[1286] After the training session, the user (worker) records feedback using a dedicated interface.

[1287] Input: Operator feedback during training session

[1288] Output: Recorded feedback data

[1289] What it does: Workers complete a training session and then use a dedicated web interface to provide feedback on the effectiveness of the script.

[1290] Step 10:

[1291] The server analyzes the collected feedback and performs re-training to further improve the generative artificial intelligence model.

[1292] Input: Recorded feedback data

[1293] Output: An improved generative artificial intelligence model

[1294] What it does: Update the generative AI model based on the feedback data, then run another epoch of training, reassessing the model's accuracy and effectiveness, and continuously optimizing it.

[1295] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1296] MODE FOR CARRYING OUT THE INVENTION

[1297] The present invention provides a system that collects the customer service scripts and know-how of highly productive workers and trains them into a generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills, and also combines it with an emotion engine that recognizes the emotions of users.Embodiments of the present invention are described in detail below.

[1298] Data collection

[1299] First, the user (high productivity worker) records voice and text data on the terminal while serving customers. The recorded voice data is the content of the conversation and response during the customer service. The user (high productivity worker) then uploads the recorded data to the server via a dedicated web interface.

[1300] Data Preprocessing

[1301] The uploaded data is preprocessed by the server, which converts the audio data into text data, removes unnecessary noise and irrelevant information, and converts different text formats into a unified format, ensuring data consistency.

[1302] emotion recognition

[1303] The acquired voice data may contain the user's emotional information. The terminal uses an emotion engine to recognize the user's emotion from the data. The emotion recognition result is used in subsequent processing.

[1304] Learning Process

[1305] The preprocessed data is supplied to the generative AI model. The server then executes a learning process based on this data, teaching the model the specific skills and effective conversation scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is fully optimized. The emotional information recognized by the emotion engine can also be taught to the generative AI model, enabling more human-like responses.

[1306] Model evaluation and improvement

[1307] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and, if there are any shortcomings, determines that the model needs to be retrained.

[1308] Script generation and dynamic adjustment

[1309] Once a high-performance talk script is generated, it is distributed to all workers. The talk script is dynamically adjusted according to the user's emotions. Specifically, the server recognizes the user's emotions in real time using an emotion engine and adjusts the talk script based on that information. For example, if the user is dissatisfied, a script with a more polite and comforting tone is used.

[1310] Gathering feedback and relearning

[1311] After the training session, the user (worker) records feedback through a dedicated interface. The server analyzes the collected feedback and performs re-training to improve the model based on the feedback. This allows the generative artificial intelligence model to be continuously improved.

[1312] Specific examples

[1313] Example 1: Collecting and learning talk scripts

[1314] 1. The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving the customer. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[1315] 2. The user uploads the recorded data to the server through a web interface.

[1316] 3. The server converts the uploaded audio data into text data and removes unnecessary noise.

[1317] 4. The preprocessed data is fed to the generative AI, where learning begins. The emotion engine also extracts the user's emotional information from the voice data, which is then used as learning data.

[1318] Example 2: Dynamic talk script generation and feedback

[1319] 1. The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[1320] 2. The user (worker) conducts a training session using the distributed script. During the training, the emotion engine recognizes the user's emotions in real time and adjusts the script accordingly.

[1321] 3. After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[1322] 4. The server analyzes the collected feedback and retrains the model if necessary.

[1323] In this way, the system of the present invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. In addition, the introduction of an emotion engine enables more personalized customer service, which will also improve customer satisfaction.

[1324] The processing flow will be explained below.

[1325] Program processing flow

[1326] Step 1: Data collection

[1327] 1. User (highly productive worker)

[1328] Record audio of talk scripts used by highly productive workers in their daily customer service work.

[1329] For example: "Hello, thank you for visiting. How can I help you today?"

[1330] 2. Terminal

[1331] Stores recorded audio data and generates a corresponding text log.

[1332] Saving example: audio_dataA.wav, text_logA.txt

[1333] 3. User (highly productive worker)

[1334] The collected data is uploaded to a server via a dedicated web interface.

[1335] Upload example: Send text_logA.txt to the server.

[1336] Step 2: Data Preprocessing

[1337] 1. Server

[1338] Convert the uploaded voice data into text data (voice recognition processing).

[1339] Example of converted text: audio_dataA.wav → "Hello, thank you for visiting. How can we help you today?"

[1340] 2. Server

[1341] Preprocessing is performed to standardize the converted text data into a uniform format.

[1342] Preprocessing example: Removing unnecessary symbols and noise data.

[1343] 3. Server

[1344] The preprocessed data is formatted as a talk script.

[1345] Formatting example: "Hello, thank you for visiting. What are you looking for today?"

[1346] Step 3: Emotion Recognition

[1347] 1. Terminal

[1348] The acquired voice data is analyzed by an emotion engine to identify the user's emotions.

[1349] Analysis example: Identifying emotions such as "joy," "anger," "sadness," and "surprise" from voice.

[1350] 2. Terminal

[1351] The identified emotion information is transmitted to a server together with the text data.

[1352] Example of transmission: Emotion information "joy" is sent to the server.

[1353] Step 4: Model training

[1354] 1. Server

[1355] The formatted talk script and the emotional information obtained from the emotion engine are fed into the generative artificial intelligence model to begin the learning process.

[1356] Example learning: "Calling customers by name can lead to more effective customer service."

[1357] 2. Server

[1358] Iteratively train the data until the model is trained.

[1359] Example iteration: Run training for 100 epochs.

[1360] Step 5: Model evaluation

[1361] 1. Server

[1362] A separate validation dataset is prepared to evaluate the performance of the trained model, and validation is performed.

[1363] Validation example: Analysis of accuracy, customer satisfaction scores.

[1364] 2. User (Administrator)

[1365] Check the evaluation results of the trained model and determine if there are any deficiencies and whether the model needs to be retrained.

[1366] Example: If the assessment score is less than 95%, decide to restudy.

[1367] Step 6: Generate and distribute scripts

[1368] 1. Server

[1369] High-performance talk scripts are generated based on successful models and distributed to all workers.

[1370] Example of a distribution script: "Thank you for visiting us, Mr. / Ms. XX. What kind of service would you like today?"

[1371] 2. Terminal

[1372] Set up a training session for all workers and role-play using the new script.

[1373] Step 7: Dynamic Adjustment

[1374] 1. Terminal

[1375] During a training session, an emotion engine is used in real time to recognize the user's emotions.

[1376] Analysis example: If the user feels "unhappy" during training, a corresponding script is recommended.

[1377] 2. Server

[1378] The talk script is dynamically adjusted based on emotional information obtained in real time.

[1379] Example of adjustment: If the user is perceived as "unhappy," change the talk script to a more polite and comforting tone.

[1380] Step 8: Gather feedback and improve

[1381] 1. User (operator)

[1382] Record feedback after training sessions using a dedicated interface.

[1383] Example feedback: "Customers responded very well to the new script, but often prefer a more casual tone."

[1384] 2. Server

[1385] Analyze the collected feedback, identify areas for improvement in the model, and retrain it.

[1386] Example of improvement: Use the collected feedback to further casualize the script.

[1387] In this way, the system of the present invention standardizes the quality of customer service scripts and improves overall productivity. Furthermore, the introduction of an emotion engine enables more personalized customer service, thereby improving customer satisfaction.

[1388] Example 2

[1389] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1390] It is necessary to uniformly impart the skills and know-how of highly productive workers to all workers, so that they can have a high level of customer service skills. There is also a need to improve customer satisfaction by recognizing users' emotions in real time and providing personalized responses. As it is difficult to effectively solve these issues using conventional methods, a new system is needed.

[1391] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1392] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for collecting feedback from workers and further improving the AI ​​model, and means for recognizing user emotions and dynamically adjusting the talk scripts based on that information. This enables all workers to have uniformly high levels of customer service skills and realizes personalized responses according to the user's emotions.

[1393] A "highly productive worker" refers to a worker who has excellent customer service skills and efficiency, and who achieves excellent results in their work.

[1394] A "talk script" refers to a document or data that compiles standard conversation flows and phrases used when serving or interacting with customers.

[1395] "Know-how" refers to practical knowledge and skills for performing specific tasks or operations.

[1396] "Preprocessing" refers to the process of removing noise and standardizing the format of data before it is fed into a generative artificial intelligence model.

[1397] "Generative AI" refers to AI that generates new data and responses based on learned data.

[1398] A "generative artificial intelligence model" refers to an algorithm or structure for prediction or generation that is constructed based on the learning results of generative artificial intelligence.

[1399] "Evaluating and improving performance" refers to verifying the accuracy and effectiveness of the generative AI model, and retraining or adjusting the model as necessary.

[1400] "High-performance talk scripts" refer to talk scripts generated by a generative artificial intelligence model that are evaluated as having high customer service skills and effectiveness.

[1401] "Feedback" refers to collecting opinions and impressions that workers gain through training sessions and on-the-job tasks.

[1402] "Recognizing user emotions" refers to analyzing and acquiring the user's emotional state from voice and text data.

[1403] "Dynamic adjustment" refers to changing the content of the talk script provided in real time according to the user's situation and emotions.

[1404] MODE FOR CARRYING OUT THE INVENTION

[1405] The present invention provides a system that collects the customer service scripts and know-how of highly productive workers and trains them into a generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills, and also combines it with an emotion engine that recognizes the emotions of users.Embodiments of the present invention are described in detail below.

[1406] Data collection

[1407] First, the user (highly productive worker) records voice and text data on the device while serving customers. Specifically, the device's recording function is used to record the conversation. For example, a script such as "Hello, thank you for visiting. What can I do for you today?" is recorded. The recorded voice data and related text data are then uploaded to the server via a dedicated web interface. Using a web browser, the user authenticates on the login screen, clicks the "File Upload" button, selects the data, and then clicks the "Send" button to complete the upload.

[1408] Data Preprocessing

[1409] The uploaded data is preprocessed by the server. First, speech recognition software (e.g., Google Cloud Speech-to-Text API) is used to convert the audio data into text data. The server sends the audio file to the speech recognition API and receives the converted text data. Then, it removes unnecessary noise and irrelevant information from the text data and standardizes the data into a standardized format using natural language processing tools (e.g., Python's NLTK library). This generates data in a standardized format.

[1410] emotion recognition

[1411] The acquired voice data may contain the user's emotional information. The terminal uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotion from this data. Specifically, the terminal inputs the voice data into the emotion recognition engine and obtains the output emotion label. This allows the emotional information to be extracted.

[1412] Learning Process

[1413] The preprocessed data is fed into a generative AI model (for example, OpenAI's GPT model). The server then runs a learning process based on this data, teaching the model the specific skills and effective conversation scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized. The emotional information recognized by the emotion engine can also be taught to the generative AI model, enabling it to respond more like a human.

[1414] Model evaluation and improvement

[1415] After training is complete, the server evaluates the performance of the generative artificial intelligence model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and determines whether the model needs to be retrained if there are any shortcomings. The evaluation results are displayed visually in a web interface, and the administrator can start retraining the model by pressing the "Retrain" button.

[1416] Script generation and dynamic adjustment

[1417] Once a high-performance talk script has been generated, it is distributed to all workers. The talk script is dynamically adjusted according to the user's emotions. Specifically, the server recognizes the user's emotions in real time using an emotion recognition engine and adjusts the talk script based on that information. For example, a prompt such as "For dissatisfied users, generate a script with a more polite and comforting tone" can be input and the generated script can be obtained.

[1418] Gathering feedback and relearning

[1419] After the training session, the user (worker) records feedback through a dedicated interface. For example, "The new script was very effective, but I thought it needed to be a little more casual." The server analyzes the collected feedback and retrains the model as needed. This allows the generative artificial intelligence model to continuously improve.

[1420] In this way, the system of the present invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. Furthermore, the introduction of an emotion engine enables more personalized customer service, which will also improve customer satisfaction.

[1421] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1422] Step 1:

[1423] The user records the conversation during the customer service session using the device's recording function. Specifically, the user presses the recording button on the device to start the conversation, and presses the button again to stop recording when finished. The input is audio data, and the output is an audio file saved on the device.

[1424] Step 2:

[1425] The user uploads the recorded audio data and associated text data to the server through a dedicated web interface. Specifically, the user opens a web browser, authenticates, clicks the "File Upload" button, selects the audio file, and clicks the "Send" button. The input is the audio file and text data, and the output is the data uploaded to the server.

[1426] Step 3:

[1427] The server converts the uploaded voice data into text data using voice recognition software (e.g., Google Cloud Speech-to-Text API). Specifically, the server sends the voice file to the API and receives the converted text data. The input is voice data, and the output is text data.

[1428] Step 4:

[1429] The server uses natural language processing tools (e.g., Python's NLTK library) to remove unnecessary noise from the converted text data and standardize the data into a format. Specifically, it analyzes the text data, removes unnecessary words and sentences, and converts them into grammatically correct sentences. The input is text data, and the output is formatted text data.

[1430] Step 5:

[1431] The device uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotion from the recorded voice data. Specifically, the voice data is input to the engine, and an emotion label is output. The input is the voice data, and the output is emotion information.

[1432] Step 6:

[1433] The server feeds the preprocessed text data and sentiment information to a generative artificial intelligence model (e.g., OpenAI's GPT model). The server inputs the dataset into the model's training script and runs training for a specified number of epochs. The input is the preprocessed data, and the output is a trained model.

[1434] Step 7:

[1435] The server uses a separate validation dataset to evaluate the performance of the trained model. Specifically, it inputs test data to the model and runs a script that calculates evaluation metrics such as precision and recall. The input is the validation dataset, and the output is the evaluation results.

[1436] Step 8:

[1437] The user (administrator) checks the evaluation results and decides whether to retrain if necessary. The evaluation results are visually displayed in the web interface, and the administrator presses the "Retrain" button to start retraining the model. The input is the evaluation results, and the output is the execution of retraining or the adoption of the model.

[1438] Step 9:

[1439] The server generates high-performance talk scripts based on the trained model and distributes them to all workers. Specifically, it inputs a prompt statement such as "Generate a new customer service talk script" into the model and retrieves the generated script. The input is the prompt statement, and the output is the generated talk script.

[1440] Step 10:

[1441] The terminal uses an emotion recognition engine to recognize the user's emotions in real time while serving the customer, and sends that information to the server. The server then dynamically adjusts the generated conversation script based on this information. For example, the terminal can input a prompt such as, "If the user is dissatisfied, generate a script with a more polite and comforting tone." The input is real-time emotional information, and the output is the adjusted conversation script.

[1442] Step 11:

[1443] After a training session, the user (worker) records their feedback through a dedicated interface. Specifically, after the training session, they press the "Feedback" button and enter their opinions and thoughts in the comments field. The input is the worker's feedback, and the output is the feedback data recorded on the server.

[1444] Step 12:

[1445] The server analyzes the collected feedback and retrains the model as needed. Specifically, it inputs the feedback data into an analysis script, identifies areas for improvement, and then retrains the model based on this. The input is the feedback data, and the output is an improved model.

[1446] This process allows the system to generate high-quality customer service scripts and provide all staff with a uniformly high level of customer service skills. The introduction of an emotion engine also enables personalized responses based on the user's real-time emotions.

[1447] (Application example 2)

[1448] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1449] In conventional customer service work, the skills and know-how of highly productive workers are individual and cannot be easily transferred to other workers. This makes it difficult for all workers to have consistently high levels of customer service skills, leading to variations in customer satisfaction. Furthermore, it is not possible to respond appropriately to customers' emotions, which can result in a decline in the quality of service.

[1450] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1451] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for analyzing the voices of the workers in real time and recognizing customer emotions, means for dynamically presenting optimal talk scripts based on the customer emotions, and means for collecting feedback from the workers and further improving the AI ​​model. This allows all workers to have uniformly high levels of customer service skills and be able to respond appropriately to customers' emotions, which is expected to improve customer satisfaction.

[1452] A "highly productive worker" is a worker who has high productivity and skills in customer service work and can increase customer satisfaction.

[1453] A "talk script" refers to a series of responses and greetings that a worker uses when serving a customer.

[1454] "Know-how" refers to specific practical methods based on the skills, knowledge, and experience of highly productive workers.

[1455] "Means of recording" refers to a method in which highly productive workers record conversations and know-how while serving customers on a terminal and save it as data that can be analyzed later.

[1456] "Preprocessing means" refers to the technical process of converting recorded data into a format that is easier to analyze and removing noise and unnecessary information.

[1457] "Generative AI" is a form of AI that learns from large amounts of data, recognizes patterns, and generates new information and responses.

[1458] "Training" is the process of feeding preprocessed data into a generative artificial intelligence model and training it based on human supervision.

[1459] "Means for evaluating and improving performance" refers to methods for evaluating whether a generative artificial intelligence model is functioning properly and providing feedback and re-training to improve performance as necessary.

[1460] The "distribution means" is a method for providing the generated high-performance talk script to all workers so that it can be used uniformly.

[1461] "Means for analyzing voice in real time" refers to technology that analyzes the voice of workers on the spot and instantly understands the content of conversations with customers.

[1462] "Means for recognizing emotions" refers to technology that analyzes and identifies a customer's emotional state from real-time voice data and facial expressions.

[1463] The "dynamic presentation means" is a method of instantly changing an appropriate talk script according to the recognized customer emotions and providing it to the worker.

[1464] "Means for collecting feedback" refers to technology that records evaluations and opinions from workers after use and uses them to improve generative artificial intelligence.

[1465] The system embodying the present invention aims to standardize the customer service skills of highly productive workers so that all workers can provide high-level customer service. This system is realized by combining multiple means.

[1466] First, the user records the conversations and interactions they have with customers using a smartphone or smart glasses, and the recorded data is uploaded to a server.

[1467] The server then preprocesses the uploaded data: the audio data is converted to text using Google Cloud Speech-to-Text, noise and unnecessary information are removed, and the text data is formatted into a unified format.

[1468] After preprocessing, the data is trained by a generative artificial intelligence model (e.g., OpenAI GPT-4). The server then uses this data to execute a learning process to generate highly productive worker skills and effective conversation scripts. This learning process also takes into account customer emotions, enabling a more human-like response.

[1469] The generated generative artificial intelligence model is then evaluated for its performance. The server evaluates the model's performance using a separate validation dataset and improves the model as necessary based on the results.

[1470] After the model has been fully optimized, this high-performance conversation script is distributed to the worker. The worker then uses the distributed script to serve customers. During this process, the worker's voice is analyzed in real time, and the customer's emotions are recognized using the Microsoft Azure Emotion API. Based on this information, the optimal conversation script is dynamically presented. For example, if a customer expresses dissatisfaction, the script displayed reads, "I'm sorry. I'd like to know more about this. What is the problem?"

[1471] After the training session, the workers record their feedback in a dedicated interface and send it to the server, which analyzes the collected feedback and retrains the generative AI model as needed to improve its quality.

[1472] These processes will ensure that all staff have a uniformly high level of customer service skills, enabling them to respond appropriately to customers' emotions, and are expected to improve customer satisfaction.

[1473] Specific examples

[1474] Prompt sentence for generative AI model

[1475] "Generate a conversation script to be used when a customer is unhappy. For example, include phrases such as, 'Sorry, customer. What is the problem?'"

[1476] In this way, with the right prompts, the generative artificial intelligence model can generate high-quality conversation scripts, ensuring that all workers deliver a consistent, high-quality experience to customers.

[1477] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1478] Step 1:

[1479] Data collection

[1480] Input: Highly productive workers use smartphones or smart glasses to record conversations and responses while serving customers.

[1481] Processing: The device temporarily stores the recorded data.

[1482] Output: Saved audio file.

[1483] Specific operation: The user activates the recording function of the smart device and starts recording.

[1484] Step 2:

[1485] Data upload

[1486] Input: The audio file saved in step 1.

[1487] Processing: The device uploads the audio file to the server.

[1488] Output: Audio data uploaded to the server.

[1489] Specific operation: After the user has finished recording, they use a dedicated application to send the data to the server.

[1490] Step 3:

[1491] Data Preprocessing

[1492] Input: Audio data uploaded to the server.

[1493] Processing: The server converts the audio data into text using Google Cloud Speech-to-Text, removing noise and unnecessary information.

[1494] Output: Preprocessed text data.

[1495] Specific operation: The server uses a speech recognition engine to analyze the voice data and converts the results into a standard format.

[1496] Step 4:

[1497] Learning with generative artificial intelligence models

[1498] Input: Preprocessed text data.

[1499] Processing: The server uses the OpenAI GPT-4 generative artificial intelligence model to learn useful talking scripts and know-how from the pre-processed data.

[1500] Output: A trained generative artificial intelligence model.

[1501] Specific operation: The server trains repeatedly for the specified number of epochs and generates the optimal model.

[1502] Step 5:

[1503] Model evaluation

[1504] Input: A trained generative artificial intelligence model and a separate validation dataset.

[1505] Processing: The server evaluates the performance of the generative artificial intelligence model using a validation dataset.

[1506] Output: Evaluation results and necessary improvements.

[1507] Specific operation: The server automatically runs the evaluation tool and generates performance metrics.

[1508] Step 6:

[1509] Talk script generation and distribution

[1510] Input: Trained model and evaluation results.

[1511] Processing: The server generates a high-performance talk script and distributes it to the worker.

[1512] Output: Talk script distributed to the worker.

[1513] Specific operation: The server automatically distributes the talk script via text message or application.

[1514] Step 7:

[1515] Real-time analytics and emotion recognition

[1516] Input: Real-time voice data of the worker.

[1517] Processing: The device uses the Microsoft Azure Emotion API to recognize customer emotions in real time.

[1518] Output: Emotion recognition result.

[1519] Specific operation: The device transmits voice data to the analysis server and acquires emotion data in real time.

[1520] Step 8:

[1521] Dynamic Script Presentation

[1522] Input: Emotion recognition results and talk script.

[1523] Processing: The server dynamically presents the optimal talk script to the worker based on the emotion recognition results.

[1524] Output: Optimized talk script.

[1525] Specific operation: The server analyzes the emotional data and displays an appropriate response on the worker's terminal.

[1526] Step 9:

[1527] Feedback collection

[1528] Input: Feedback from the worker.

[1529] Processing: The terminal sends the collected feedback to the server.

[1530] Output: Feedback data.

[1531] Specific operation: The user inputs feedback using a dedicated interface.

[1532] Step 10:

[1533] Retraining the Model

[1534] Input: Feedback data and an existing generative artificial intelligence model.

[1535] Processing: The server retrains the generative artificial intelligence model based on the feedback data.

[1536] Output: An improved generative artificial intelligence model.

[1537] What happens: The server analyzes the feedback and retrains the model with the new data.

[1538] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1539] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1540] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1541] [Fourth embodiment]

[1542] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1543] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1544] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1545] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1546] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1547] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1548] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1549] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1550] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1551] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1552] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1553] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1554] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1555] MODE FOR CARRYING OUT THE INVENTION

[1556] The present invention provides a system that collects customer service scripts and know-how from highly productive workers and trains them using generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills. An embodiment of the present invention is described in detail below.

[1557] Data collection

[1558] First, the user (high productivity worker) records voice and text data on the terminal while serving customers. The recorded voice data is the content of the conversation and response during the customer service. The user (high productivity worker) then uploads the recorded data to the server via a dedicated web interface.

[1559] Data Preprocessing

[1560] The uploaded data is preprocessed by the server, which converts the audio data into text data, removes unnecessary noise and irrelevant information, and converts different text formats into a unified format, ensuring data consistency.

[1561] Learning Process

[1562] The preprocessed data is fed to a generative artificial intelligence model. The server then executes a learning process based on the data, teaching the model the specific skills and effective speech scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized.

[1563] Model evaluation and improvement

[1564] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and, if there are any shortcomings, determines that the model needs to be retrained.

[1565] Script generation and distribution

[1566] After generating a high-performance talk script, the server distributes it to all workers. Specifically, users (workers) use the distributed talk script to conduct training sessions. In these sessions, role-playing using the new script is conducted, simulating the application of the script to real customer service work.

[1567] Gathering feedback and relearning

[1568] After the training session, the user (worker) records feedback through a dedicated interface. The server analyzes the collected feedback and performs re-training to improve the model based on the feedback. This allows the generative artificial intelligence model to be continuously improved.

[1569] Specific examples

[1570] Example 1: Collecting and learning talk scripts

[1571] 1. The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving the customer. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[1572] 2. The user uploads the recorded data to the server through a web interface.

[1573] 3. The server converts the uploaded audio data into text data and removes unnecessary noise.

[1574] 4. The preprocessed data is fed into the generative artificial intelligence, which begins learning.

[1575] Example 2: Training and feedback

[1576] 1. The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[1577] 2. The user (operator) conducts a training session using the distributed script.

[1578] 3. After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[1579] 4. The server analyzes the collected feedback and retrains the model if necessary.

[1580] This invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. Furthermore, through continuous feedback and re-learning, the generative AI model can continue to provide the latest, high-quality scripts.

[1581] The processing flow will be explained below.

[1582] Program processing flow

[1583] Step 1: Data collection

[1584] 1. User (highly productive worker)

[1585] Record audio of talk scripts used by highly productive workers in their daily customer service work.

[1586] For example: "Hello, thank you for visiting. How can I help you today?"

[1587] 2. Terminal

[1588] Stores recorded audio data and generates a corresponding text log.

[1589] Saving example: audio_dataA.wav, text_logA.txt

[1590] 3. User (highly productive worker)

[1591] The collected data is uploaded to a server via a dedicated web interface.

[1592] Upload example: Send text_logA.txt to the server.

[1593] Step 2: Data Preprocessing

[1594] 1. Server

[1595] Convert the uploaded voice data into text data (voice recognition processing).

[1596] Example of converted text: audio_dataA.wav → "Hello, and welcome to SoftBank. How can we help you today?"

[1597] 2. Server

[1598] Preprocessing is performed to standardize the converted text data into a uniform format.

[1599] Preprocessing example: Removing unnecessary symbols and noise data.

[1600] 3. Server

[1601] The preprocessed data is formatted as a talk script.

[1602] Formatting example: "Hello, welcome to SoftBank. What can I do for you today?"

[1603] Step 3: Model training

[1604] 1. Server

[1605] The formatted talk script is fed into a generative artificial intelligence model to begin the learning process.

[1606] Example learning: "Calling customers by name can lead to more effective customer service."

[1607] 2. Server

[1608] Iteratively train the data until the model is trained.

[1609] Example iteration: Run training for 100 epochs.

[1610] Step 4: Model evaluation

[1611] 1. Server

[1612] A separate validation dataset is prepared to evaluate the performance of the trained model, and validation is performed.

[1613] Validation example: Analysis of accuracy, customer satisfaction scores.

[1614] 2. User (Administrator)

[1615] Check the evaluation results of the trained model and determine if there are any deficiencies and whether the model needs to be retrained.

[1616] Example: If the assessment score is less than 95%, decide to restudy.

[1617] Step 5: Generate and distribute scripts

[1618] 1. Server

[1619] High-performance talk scripts are generated based on successful models and distributed to all workers.

[1620] Example of a distribution script: "Thank you for visiting us, Mr. / Ms. XX. What kind of service would you like today?"

[1621] 2. Terminal

[1622] Set up a training session for all workers and role-play using the new script.

[1623] Step 6: Gather feedback and improve

[1624] 1. User (operator)

[1625] Record feedback after training sessions using a dedicated interface.

[1626] Example feedback: "Customers responded very well to the new script, but often prefer a more casual tone."

[1627] 2. Server

[1628] Analyze the collected feedback, identify areas for improvement in the model, and retrain it.

[1629] Example of improvement: Use the collected feedback to further casualize the script.

[1630] Example 1

[1631] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1632] In conventional customer service operations, it was difficult to effectively share the talk scripts and know-how of highly productive workers with other workers. As a result, there was a large difference in customer service skills among individuals, resulting in an inconsistent overall service quality. Furthermore, the process of improving customer service scripts through continuous training and feedback was not carried out efficiently, making it difficult to achieve sustained operational improvement. A system was needed to solve these issues and enable all workers to acquire and maintain a high level of customer service skills.

[1633] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1634] In this invention, the server includes: means for recording the talk scripts and know-how of highly productive workers; means for uploading the recorded talk scripts and know-how to the server through a dedicated interface; means for converting the uploaded voice data into text data and removing unnecessary noise and irrelevant information; means for converting the preprocessed data into a unified format; means for training a generative artificial intelligence (AI) based on the preprocessed data; means for evaluating the performance of the AI ​​model using a separate validation dataset and identifying areas for improvement; means for generating high-performance talk scripts and distributing them to all workers; means for workers to record feedback after training sessions; and means for analyzing the collected feedback and retraining the AI ​​model to continuously improve it. This enables all workers to have a uniformly high level of customer service skills, facilitating continuous improvement and improving overall service quality.

[1635] A "highly productive worker" is a worker who has outstanding skills and know-how in customer service work and demonstrates higher productivity and effectiveness than other workers.

[1636] A "talk script" is a set of predetermined conversational content and phrases used during customer service.

[1637] "Know-how" refers to practical knowledge based on specialized knowledge, skills, and experience required to effectively carry out specific tasks or operations.

[1638] A "dedicated interface" is a specific operating screen or system that allows users to input, manage, and transmit data.

[1639] A "server" is a computer system that stores and processes data via a network, and plays a central role in this system.

[1640] "Preprocessing" refers to a series of processing operations performed on raw data to prepare it in a form suitable for analysis and learning.

[1641] "Generative AI" is a type of AI that has the ability to learn patterns from large amounts of data and generate and respond to new data.

[1642] "High-performance talk scripts" refer to standard, high-quality conversation phrases and content generated by generative artificial intelligence.

[1643] "Feedback" refers to the evaluations and opinions provided by workers after training sessions or actual work performance, which are used to improve the system.

[1644] A "training session" is a training activity in which workers simulate real-world work scenarios using pre-prepared scripts.

[1645] "Retraining" is the process of retraining a generative artificial intelligence model based on new data and feedback collected to improve its performance.

[1646] "Audio data" refers to data that records the user's voice, and is used to record the contents of the conversation.

[1647] "Text data" is voice data converted into text information and is used for analysis and learning.

[1648] "Noise" refers to unnecessary information or noise that should not be included in the data being analyzed.

[1649] A "format" refers to the form or structure that allows data to be handled uniformly.

[1650] A "validation dataset" is data prepared separately from the training data and used to evaluate the performance of a generative artificial intelligence model.

[1651] MODE FOR CARRYING OUT THE INVENTION

[1652] The present invention provides a system that collects customer service scripts and know-how from highly productive workers and trains them in a generative artificial intelligence (AI) system, thereby enabling all workers to have uniformly high levels of customer service skills. The following describes an embodiment of the present invention.

[1653] Data collection

[1654] The user (highly productive worker) records voice and text data during customer service operations on a terminal. This terminal is a device with a recording function, such as a smartphone or tablet. The recorded voice data specifically includes the content of the conversation during customer service, such as "Hello, thank you for visiting our store. What can I do for you today?" The user uploads the recorded data to a server via a dedicated web interface.

[1655] Data Preprocessing

[1656] The server converts the uploaded voice data into text data using speech recognition software such as the Google Cloud Speech-to-Text API. The server then uses text analysis tools such as the NLTK library to remove unnecessary noise and irrelevant information. This ensures data consistency and converts it into a unified format (e.g., JSON).

[1657] Learning Process

[1658] The preprocessed data is fed into a generative artificial intelligence model (e.g., OpenAI's GPT-3). The server executes the learning process based on the specified number of epochs (e.g., 100 epochs). During the learning process, the model learns the specific skills and effective talking scripts of highly productive workers. This process is distributed using a high-performance GPU cluster.

[1659] Model evaluation and improvement

[1660] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. Metrics such as F1 score and accuracy are used in the evaluation. The user (administrator) can check the evaluation results through the dashboard and decide whether to retrain the model if there are any shortcomings.

[1661] Script generation and distribution

[1662] Once a high-performance talk script has been generated, the server distributes it to all workers. Specifically, the server sends notifications via email or a dedicated app. For example, a talk script such as "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?" may be distributed. Users (workers) use the distributed script to conduct training sessions and role-play.

[1663] Gathering feedback and relearning

[1664] After the training session, the user (worker) records their impressions and suggestions for improvement through a dedicated feedback interface (e.g., a web form). For example, specific opinions such as "The new script was very effective, but I thought it needed to be a little more casual in its expressions" can be recorded. The server analyzes the collected feedback using natural language processing technology and reflects it in the next re-training. This allows the generative AI model to be continuously improved.

[1665] Specific examples

[1666] Example 1: Collecting and learning talk scripts

[1667] The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving customers. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[1668] The user uploads the recorded data to a server through a web interface.

[1669] The server converts the uploaded voice data into text data and removes unnecessary noise.

[1670] The pre-processed data is fed into the generative artificial intelligence, which begins learning.

[1671] Example 2: Training and feedback

[1672] The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[1673] The user (operator) conducts a training session using the distributed script.

[1674] After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[1675] The server analyzes the collected feedback and retrains the model if necessary.

[1676] Prompt Sentence Examples

[1677] "We want to train the AI ​​based on customer service scripts written by highly productive workers. We would like you to record the following conversations and carry out the training process:

[1678] "Hello, thank you for coming. How can I help you today?"

[1679] "Thank you for visiting us, Mr. / Ms. X. What kind of service would you like today?"

[1680] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1681] Step 1: Data collection

[1682] The user (highly productive worker) records conversations during customer service duties on a device. For example, using a recording app on a smartphone, the user can record a conversation that goes something like, "Hello, thank you for visiting our store. What can I do for you today?" Once the recording is complete, the user uploads the data to a server via a dedicated web interface.

[1683] Input: Voice data during customer service

[1684] Output: Recording data is uploaded to the server

[1685] How it works: Record audio using a recording app on your smartphone and upload the file to a server using a dedicated web interface. The web interface is an application that runs on a browser.

[1686] Step 2: Data Preprocessing

[1687] The server converts the uploaded audio data into text data. It uses speech recognition software, such as the Google Cloud Speech-to-Text API, to convert the recording into text. The server then uses a text analysis tool, such as NLTK, to remove unnecessary noise and irrelevant information from the data. It also converts the text data into a standardized format, such as JSON.

[1688] Input: Uploaded audio data

[1689] Output: Preprocessed text data

[1690] What it does: The server runs speech recognition software, analyzes the resulting text data, removes noise, and converts the data into a unified format.

[1691] Step 3: The learning process

[1692] The server supplies the preprocessed text data to a generative artificial intelligence model (e.g., OpenAI's GPT-3). The learning process begins based on a specified number of epochs (e.g., 100 epochs). The model learns the specific skills and speech scripts of highly productive workers. This process is distributed using a high-performance GPU cluster.

[1693] Input: Preprocessed text data

[1694] Output: A trained generative artificial intelligence model

[1695] Specific operation: The server inputs text data into the generative AI model and performs learning for the specified number of epochs. The learning progress is recorded in the server's log.

[1696] Step 4: Model evaluation and improvement

[1697] The server evaluates the performance of the generative AI model using a separate validation dataset. The effectiveness of the model is confirmed based on evaluation indicators such as F1 score and accuracy. The user (administrator) can check the evaluation results via the dashboard and decide whether to retrain the model if any shortcomings are found.

[1698] Input: trained model, validation dataset

[1699] Output: Model performance evaluation results

[1700] Specific operation: The server inputs the validation data into the model and displays the obtained score on the dashboard. The administrator can check the results on the dashboard and instruct the model to be retrained if necessary.

[1701] Step 5: Generate and distribute scripts

[1702] The server generates high-performance conversation scripts and distributes them to all workers. Specifically, the server sends notifications via email or a dedicated app. For example, a conversation script such as "Thank you for visiting our store, Mr. / Ms. X. What kind of service would you like today?" is generated and distributed. Users (workers) use the distributed scripts to conduct training sessions and role-play.

[1703] Input: A trained model

[1704] Output: Generated talk script

[1705] How it works: The server uses the trained model to generate a talk script and sends it to the worker via email or a dedicated app. The worker receives the script and practices it during the training session.

[1706] Step 6: Gather feedback and retrain

[1707] After the training session, the user (worker) records their impressions and suggestions for improvement through a dedicated feedback interface (e.g., a web form). The server analyzes the collected feedback using natural language processing technology and continuously improves the generative AI model by retraining it.

[1708] Input: Feedback data

[1709] Output: An improved generative artificial intelligence model

[1710] Specific operation: The worker enters feedback into a web form and submits it. The server analyzes the feedback data and retrains the model as necessary. The retrained model is used to generate a new talk script.

[1711] (Application example 1)

[1712] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1713] In conventional customer service work, there was a large variation in the customer service skills and experience of individual workers, resulting in uneven customer satisfaction and work efficiency. Furthermore, it took time and effort for workers to learn new customer service scripts, making it difficult to consistently provide high-quality service. Furthermore, there was no system in place to provide effective customer service phrases in real time, which sometimes led to inappropriate responses during customer service.

[1714] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1715] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for collecting feedback from workers and further improving the AI ​​model, and means for the AI ​​to suggest customer service phrases in real time when a store clerk converses with a customer. This allows all workers to have uniformly high levels of customer service skills and enables the provision of effective customer service phrases in real time.

[1716] "Highly productive workers" refer to workers who have high productivity and excellent skills in customer service work.

[1717] A "talk script" refers to a series of phrases and responses used in customer service.

[1718] "Know-how" refers to techniques, knowledge, and methods that are effective in a specific business or task.

[1719] "Preprocessing" refers to the process of removing unnecessary noise and irrelevant information from recorded data and standardizing it into an appropriate format.

[1720] "Generative AI" refers to AI that has the ability to create new data and scripts based on input data.

[1721] "Performance evaluation of a generative artificial intelligence model" refers to the act of verifying whether the output of a generative artificial intelligence model reaches the expected level.

[1722] A "high-performance talk script" refers to a series of effective, high-quality customer service phrases optimized using generative artificial intelligence.

[1723] "Means of suggesting conversation phrases in real time" refers to a system in which generative artificial intelligence instantly suggests the most appropriate phrases when a store clerk is conversing with a customer.

[1724] The present invention is a system that collects customer service scripts and know-how from highly productive workers, trains them into a generative artificial intelligence model, and enables all workers to acquire high-level customer service skills. Specific embodiments for carrying out the invention are described below.

[1725] Data collection

[1726] First, the user (highly productive worker) records conversations during customer service using the recording function of their smartphone. The user then uploads the recorded audio data to a server via a dedicated web interface. The audio data includes the conversations and responses made during customer service.

[1727] Data Preprocessing

[1728] Once the server receives the uploaded audio data, it converts it into text using the SpeechRecognition library. It then removes unnecessary noise and irrelevant information and organizes the text data into a unified format. This preprocessing process ensures consistent data suitable for analysis.

[1729] Learning Process

[1730] The preprocessed data is fed to a generative AI model (e.g., GPT-4). The server then executes a model learning process based on this data, teaching the model the specific skills and effective speech scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized.

[1731] Model evaluation and improvement

[1732] After the training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model, and the user (administrator) can review the evaluation results and determine whether the model needs to be retrained if there are any shortcomings.

[1733] Script generation and distribution

[1734] After generating a high-performance talk script, the server distributes it to all workers. The users (workers) then use the distributed talk script to conduct training sessions. In these sessions, role-playing using the new script is conducted, simulating the application of the script to real customer service tasks.

[1735] Real-time phrase suggestions

[1736] During customer service, the server uses generative artificial intelligence to suggest optimal response phrases in real time. The user (worker) receives these in real time on their smartphone and uses them in conversation with the customer. This enables high-quality responses that are immediately suited to the situation.

[1737] Gathering feedback and relearning

[1738] After the training session, the user (worker) records feedback using a dedicated interface, and the server analyzes this feedback and performs re-training to further improve the generative artificial intelligence model.

[1739] Specific examples

[1740] For example, imagine a scenario where a new sales associate greets a customer with, "Hello, thank you for your inquiry. How can I help you today?" This conversation is recorded on a smartphone and uploaded through an application. The generative AI model analyzes this data and suggests an optimized script.

[1741] Example prompts for generative AI models

[1742] An example prompt is:

[1743] Prompt: Please optimize the following customer service script.

[1744] input:

[1745] "Hello, thank you for coming. How can I help you today?"

[1746] the goal:

[1747] 1. Address customers in a bright, friendly tone.

[1748] 2. Include questions to quickly understand the customer's needs.

[1749] Example output:

[1750] "Hello! Welcome. How can I help you today?"

[1751] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1752] Step 1:

[1753] The user (highly productive worker) records conversations while serving customers using their smartphone.

[1754] Input: Recording of voice conversations using a smartphone

[1755] Output: Recorded audio data file (e.g. .wav)

[1756] How it works: Use a dedicated application on your smartphone to record audio data through the microphone. Once the recording is complete, press the save button to create a data file.

[1757] Step 2:

[1758] Users upload the recorded audio data to the server via a dedicated web interface.

[1759] Input: Recorded audio data file

[1760] Output: Audio data file stored on the server

[1761] Specific operations: Access the web interface, press the upload button to select a file, and press the send button to transfer the audio data to the server.

[1762] Step 3:

[1763] The server receives the uploaded voice data and converts it into text data using the SpeechRecognition library.

[1764] Input: Uploaded audio data file

[1765] Output: Text data converted from audio data

[1766] Specific operation: Analyzes the audio file using the SpeechRecognition library and converts the audio signal into a string. The converted text data is temporarily saved.

[1767] Step 4:

[1768] The server preprocesses the text data, removing unnecessary noise and irrelevant information, and arranging it into a unified format.

[1769] Input: Converted text data

[1770] Output: Preprocessed clean text data

[1771] Specific operation: Using noise filtering algorithms and regular expressions, unnecessary parts are removed from text data and it is formatted into a unified format.

[1772] Step 5:

[1773] The pre-processed data is fed into a generative artificial intelligence model to begin the learning process.

[1774] Input: Preprocessed text data

[1775] Output: Trained generative AI model (initial version)

[1776] What it does: It feeds data to a generative AI model (e.g., GPT-4) and trains it for a specified number of epochs. The model learns based on a specific conversation script and response style.

[1777] Step 6:

[1778] The server evaluates the model using a separate validation dataset to evaluate the performance of the trained model.

[1779] Input: trained model and validation dataset

[1780] Output: Model performance evaluation results (accuracy, effectiveness, etc.)

[1781] What it does: It evaluates the model output using a validation dataset to measure accuracy and effectiveness, and stores the results in a database.

[1782] Step 7:

[1783] The user (administrator) checks the evaluation results and, if there are any shortcomings, instructs the model to be retrained.

[1784] Input: Model performance evaluation results

[1785] Output: Retraining instructions or model approval

[1786] Specific behavior: The administrator reviews the assessment results and determines whether retraining is necessary. If necessary, the administrator triggers the retraining process.

[1787] Step 8:

[1788] After the high-performance talk script is generated, the server distributes it to all workers.

[1789] Input: The final trained generative artificial intelligence model

[1790] Output: Distributed high performance talk script

[1791] Specific operation: The server sends the script to each worker's terminal and instructs them to conduct a training session using the new script.

[1792] Step 9:

[1793] After the training session, the user (worker) records feedback using a dedicated interface.

[1794] Input: Operator feedback during training session

[1795] Output: Recorded feedback data

[1796] What it does: Workers complete a training session and then use a dedicated web interface to provide feedback on the effectiveness of the script.

[1797] Step 10:

[1798] The server analyzes the collected feedback and performs re-training to further improve the generative artificial intelligence model.

[1799] Input: Recorded feedback data

[1800] Output: An improved generative artificial intelligence model

[1801] What it does: Update the generative AI model based on the feedback data, then run another epoch of training, reassessing the model's accuracy and effectiveness, and continuously optimizing it.

[1802] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1803] MODE FOR CARRYING OUT THE INVENTION

[1804] The present invention provides a system that collects the customer service scripts and know-how of highly productive workers and trains them into a generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills, and also combines it with an emotion engine that recognizes the emotions of users.Embodiments of the present invention are described in detail below.

[1805] Data collection

[1806] First, the user (high productivity worker) records voice and text data on the terminal while serving customers. The recorded voice data is the content of the conversation and response during the customer service. The user (high productivity worker) then uploads the recorded data to the server via a dedicated web interface.

[1807] Data Preprocessing

[1808] The uploaded data is preprocessed by the server, which converts the audio data into text data, removes unnecessary noise and irrelevant information, and converts different text formats into a unified format, ensuring data consistency.

[1809] emotion recognition

[1810] The acquired voice data may contain the user's emotional information. The terminal uses an emotion engine to recognize the user's emotion from the data. The emotion recognition result is used in subsequent processing.

[1811] Learning Process

[1812] The preprocessed data is supplied to the generative AI model. The server then executes a learning process based on this data, teaching the model the specific skills and effective conversation scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is fully optimized. The emotional information recognized by the emotion engine can also be taught to the generative AI model, enabling more human-like responses.

[1813] Model evaluation and improvement

[1814] After training is complete, the server evaluates the performance of the generative AI model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and, if there are any shortcomings, determines that the model needs to be retrained.

[1815] Script generation and dynamic adjustment

[1816] Once a high-performance talk script is generated, it is distributed to all workers. The talk script is dynamically adjusted according to the user's emotions. Specifically, the server recognizes the user's emotions in real time using an emotion engine and adjusts the talk script based on that information. For example, if the user is dissatisfied, a script with a more polite and comforting tone is used.

[1817] Gathering feedback and relearning

[1818] After the training session, the user (worker) records feedback through a dedicated interface. The server analyzes the collected feedback and performs re-training to improve the model based on the feedback. This allows the generative artificial intelligence model to be continuously improved.

[1819] Specific examples

[1820] Example 1: Collecting and learning talk scripts

[1821] 1. The user (high productivity worker) uses the recording function of the terminal to record the conversation while serving the customer. For example, a script such as "Hello, thank you for visiting our store. What can I do for you today?" is recorded.

[1822] 2. The user uploads the recorded data to the server through a web interface.

[1823] 3. The server converts the uploaded audio data into text data and removes unnecessary noise.

[1824] 4. The preprocessed data is fed to the generative AI, where learning begins. The emotion engine also extracts the user's emotional information from the voice data, which is then used as learning data.

[1825] Example 2: Dynamic talk script generation and feedback

[1826] 1. The server generates a high-performance conversation script based on the trained model and distributes it to all workers. For example, a script might be distributed that reads, "Thank you for visiting Mr. / Ms. X. What kind of service would you like today?"

[1827] 2. The user (worker) conducts a training session using the distributed script. During the training, the emotion engine recognizes the user's emotions in real time and adjusts the script accordingly.

[1828] 3. After the training session, users record their feedback through a dedicated interface, such as, "The new script was very effective, but I think it needed to be a little more casual."

[1829] 4. The server analyzes the collected feedback and retrains the model if necessary.

[1830] In this way, the system of the present invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. In addition, the introduction of an emotion engine enables more personalized customer service, which will also improve customer satisfaction.

[1831] The processing flow will be explained below.

[1832] Program processing flow

[1833] Step 1: Data collection

[1834] 1. User (highly productive worker)

[1835] Record audio of talk scripts used by highly productive workers in their daily customer service work.

[1836] For example: "Hello, thank you for visiting. How can I help you today?"

[1837] 2. Terminal

[1838] Stores recorded audio data and generates a corresponding text log.

[1839] Saving example: audio_dataA.wav, text_logA.txt

[1840] 3. User (highly productive worker)

[1841] The collected data is uploaded to a server via a dedicated web interface.

[1842] Upload example: Send text_logA.txt to the server.

[1843] Step 2: Data Preprocessing

[1844] 1. Server

[1845] Convert the uploaded voice data into text data (voice recognition processing).

[1846] Example of converted text: audio_dataA.wav → "Hello, thank you for visiting. How can we help you today?"

[1847] 2. Server

[1848] Preprocessing is performed to standardize the converted text data into a uniform format.

[1849] Preprocessing example: Removing unnecessary symbols and noise data.

[1850] 3. Server

[1851] The preprocessed data is formatted as a talk script.

[1852] Formatting example: "Hello, thank you for visiting. What are you looking for today?"

[1853] Step 3: Emotion Recognition

[1854] 1. Terminal

[1855] The acquired voice data is analyzed by an emotion engine to identify the user's emotions.

[1856] Analysis example: Identifying emotions such as "joy," "anger," "sadness," and "surprise" from voice.

[1857] 2. Terminal

[1858] The identified emotion information is transmitted to a server together with the text data.

[1859] Example of transmission: Emotion information "joy" is sent to the server.

[1860] Step 4: Model training

[1861] 1. Server

[1862] The formatted talk script and the emotional information obtained from the emotion engine are fed into the generative artificial intelligence model to begin the learning process.

[1863] Example learning: "Calling customers by name can lead to more effective customer service."

[1864] 2. Server

[1865] Iteratively train the data until the model is trained.

[1866] Example iteration: Run training for 100 epochs.

[1867] Step 5: Model evaluation

[1868] 1. Server

[1869] A separate validation dataset is prepared to evaluate the performance of the trained model, and validation is performed.

[1870] Validation example: Analysis of accuracy, customer satisfaction scores.

[1871] 2. User (Administrator)

[1872] Check the evaluation results of the trained model and determine if there are any deficiencies and whether the model needs to be retrained.

[1873] Example: If the assessment score is less than 95%, decide to restudy.

[1874] Step 6: Generate and distribute scripts

[1875] 1. Server

[1876] High-performance talk scripts are generated based on successful models and distributed to all workers.

[1877] Example of a distribution script: "Thank you for visiting us, Mr. / Ms. XX. What kind of service would you like today?"

[1878] 2. Terminal

[1879] Set up a training session for all workers and role-play using the new script.

[1880] Step 7: Dynamic Adjustment

[1881] 1. Terminal

[1882] During a training session, an emotion engine is used in real time to recognize the user's emotions.

[1883] Analysis example: If the user feels "unhappy" during training, a corresponding script is recommended.

[1884] 2. Server

[1885] The talk script is dynamically adjusted based on emotional information obtained in real time.

[1886] Example of adjustment: If the user is perceived as "unhappy," change the talk script to a more polite and comforting tone.

[1887] Step 8: Gather feedback and improve

[1888] 1. User (operator)

[1889] Record feedback after training sessions using a dedicated interface.

[1890] Example feedback: "Customers responded very well to the new script, but often prefer a more casual tone."

[1891] 2. Server

[1892] Analyze the collected feedback, identify areas for improvement in the model, and retrain it.

[1893] Example of improvement: Use the collected feedback to further casualize the script.

[1894] In this way, the system of the present invention standardizes the quality of customer service scripts and improves overall productivity. Furthermore, the introduction of an emotion engine enables more personalized customer service, thereby improving customer satisfaction.

[1895] Example 2

[1896] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1897] It is necessary to uniformly impart the skills and know-how of highly productive workers to all workers, so that they can have a high level of customer service skills. There is also a need to improve customer satisfaction by recognizing users' emotions in real time and providing personalized responses. As it is difficult to effectively solve these issues using conventional methods, a new system is needed.

[1898] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1899] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for collecting feedback from workers and further improving the AI ​​model, and means for recognizing user emotions and dynamically adjusting the talk scripts based on that information. This enables all workers to have uniformly high levels of customer service skills and realizes personalized responses according to the user's emotions.

[1900] A "highly productive worker" refers to a worker who has excellent customer service skills and efficiency, and who achieves excellent results in their work.

[1901] A "talk script" refers to a document or data that compiles standard conversation flows and phrases used when serving or interacting with customers.

[1902] "Know-how" refers to practical knowledge and skills for performing specific tasks or operations.

[1903] "Preprocessing" refers to the process of removing noise and standardizing the format of data before it is fed into a generative artificial intelligence model.

[1904] "Generative AI" refers to AI that generates new data and responses based on learned data.

[1905] A "generative artificial intelligence model" refers to an algorithm or structure for prediction or generation that is constructed based on the learning results of generative artificial intelligence.

[1906] "Evaluating and improving performance" refers to verifying the accuracy and effectiveness of the generative AI model, and retraining or adjusting the model as necessary.

[1907] "High-performance talk scripts" refer to talk scripts generated by a generative artificial intelligence model that are evaluated as having high customer service skills and effectiveness.

[1908] "Feedback" refers to collecting opinions and impressions that workers gain through training sessions and on-the-job tasks.

[1909] "Recognizing user emotions" refers to analyzing and acquiring the user's emotional state from voice and text data.

[1910] "Dynamic adjustment" refers to changing the content of the talk script provided in real time according to the user's situation and emotions.

[1911] MODE FOR CARRYING OUT THE INVENTION

[1912] The present invention provides a system that collects the customer service scripts and know-how of highly productive workers and trains them into a generative artificial intelligence, thereby enabling all workers to have uniformly high levels of customer service skills, and also combines it with an emotion engine that recognizes the emotions of users.Embodiments of the present invention are described in detail below.

[1913] Data collection

[1914] First, the user (highly productive worker) records voice and text data on the device while serving customers. Specifically, the device's recording function is used to record the conversation. For example, a script such as "Hello, thank you for visiting. What can I do for you today?" is recorded. The recorded voice data and related text data are then uploaded to the server via a dedicated web interface. Using a web browser, the user authenticates on the login screen, clicks the "File Upload" button, selects the data, and then clicks the "Send" button to complete the upload.

[1915] Data Preprocessing

[1916] The uploaded data is preprocessed by the server. First, speech recognition software (e.g., Google Cloud Speech-to-Text API) is used to convert the audio data into text data. The server sends the audio file to the speech recognition API and receives the converted text data. Then, it removes unnecessary noise and irrelevant information from the text data and standardizes the data into a standardized format using natural language processing tools (e.g., Python's NLTK library). This generates data in a standardized format.

[1917] emotion recognition

[1918] The acquired voice data may contain the user's emotional information. The terminal uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotion from this data. Specifically, the terminal inputs the voice data into the emotion recognition engine and obtains the output emotion label. This allows the emotional information to be extracted.

[1919] Learning Process

[1920] The preprocessed data is fed into a generative AI model (for example, OpenAI's GPT model). The server then runs a learning process based on this data, teaching the model the specific skills and effective conversation scripts of highly productive workers. The learning process is repeated for a specified number of epochs until the model is sufficiently optimized. The emotional information recognized by the emotion engine can also be taught to the generative AI model, enabling it to respond more like a human.

[1921] Model evaluation and improvement

[1922] After training is complete, the server evaluates the performance of the generative artificial intelligence model using a separate validation dataset. The evaluation results are used to determine the effectiveness and practicality of the model. The user (administrator) checks the evaluation results and determines whether the model needs to be retrained if there are any shortcomings. The evaluation results are displayed visually in a web interface, and the administrator can start retraining the model by pressing the "Retrain" button.

[1923] Script generation and dynamic adjustment

[1924] Once a high-performance talk script has been generated, it is distributed to all workers. The talk script is dynamically adjusted according to the user's emotions. Specifically, the server recognizes the user's emotions in real time using an emotion recognition engine and adjusts the talk script based on that information. For example, a prompt such as "For dissatisfied users, generate a script with a more polite and comforting tone" can be input and the generated script can be obtained.

[1925] Gathering feedback and relearning

[1926] After the training session, the user (worker) records feedback through a dedicated interface. For example, "The new script was very effective, but I thought it needed to be a little more casual." The server analyzes the collected feedback and retrains the model as needed. This allows the generative artificial intelligence model to continuously improve.

[1927] In this way, the system of the present invention can standardize the quality of customer service scripts, which is expected to improve overall productivity. Furthermore, the introduction of an emotion engine enables more personalized customer service, which will also improve customer satisfaction.

[1928] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1929] Step 1:

[1930] The user records the conversation during the customer service session using the device's recording function. Specifically, the user presses the recording button on the device to start the conversation, and presses the button again to stop recording when finished. The input is audio data, and the output is an audio file saved on the device.

[1931] Step 2:

[1932] The user uploads the recorded audio data and associated text data to the server through a dedicated web interface. Specifically, the user opens a web browser, authenticates, clicks the "File Upload" button, selects the audio file, and clicks the "Send" button. The input is the audio file and text data, and the output is the data uploaded to the server.

[1933] Step 3:

[1934] The server converts the uploaded voice data into text data using voice recognition software (e.g., Google Cloud Speech-to-Text API). Specifically, the server sends the voice file to the API and receives the converted text data. The input is voice data, and the output is text data.

[1935] Step 4:

[1936] The server uses natural language processing tools (e.g., Python's NLTK library) to remove unnecessary noise from the converted text data and standardize the data into a format. Specifically, it analyzes the text data, removes unnecessary words and sentences, and converts them into grammatically correct sentences. The input is text data, and the output is formatted text data.

[1937] Step 5:

[1938] The device uses an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotion from the recorded voice data. Specifically, the voice data is input to the engine, and an emotion label is output. The input is the voice data, and the output is emotion information.

[1939] Step 6:

[1940] The server feeds the preprocessed text data and sentiment information to a generative artificial intelligence model (e.g., OpenAI's GPT model). The server inputs the dataset into the model's training script and runs training for a specified number of epochs. The input is the preprocessed data, and the output is a trained model.

[1941] Step 7:

[1942] The server uses a separate validation dataset to evaluate the performance of the trained model. Specifically, it inputs test data to the model and runs a script that calculates evaluation metrics such as precision and recall. The input is the validation dataset, and the output is the evaluation results.

[1943] Step 8:

[1944] The user (administrator) checks the evaluation results and decides whether to retrain if necessary. The evaluation results are visually displayed in the web interface, and the administrator presses the "Retrain" button to start retraining the model. The input is the evaluation results, and the output is the execution of retraining or the adoption of the model.

[1945] Step 9:

[1946] The server generates high-performance talk scripts based on the trained model and distributes them to all workers. Specifically, it inputs a prompt statement such as "Generate a new customer service talk script" into the model and retrieves the generated script. The input is the prompt statement, and the output is the generated talk script.

[1947] Step 10:

[1948] The terminal uses an emotion recognition engine to recognize the user's emotions in real time while serving the customer, and sends that information to the server. The server then dynamically adjusts the generated conversation script based on this information. For example, the terminal can input a prompt such as, "If the user is dissatisfied, generate a script with a more polite and comforting tone." The input is real-time emotional information, and the output is the adjusted conversation script.

[1949] Step 11:

[1950] After a training session, the user (worker) records their feedback through a dedicated interface. Specifically, after the training session, they press the "Feedback" button and enter their opinions and thoughts in the comments field. The input is the worker's feedback, and the output is the feedback data recorded on the server.

[1951] Step 12:

[1952] The server analyzes the collected feedback and retrains the model as needed. Specifically, it inputs the feedback data into an analysis script, identifies areas for improvement, and then retrains the model based on this. The input is the feedback data, and the output is an improved model.

[1953] This process allows the system to generate high-quality customer service scripts and provide all staff with a uniformly high level of customer service skills. The introduction of an emotion engine also enables personalized responses based on the user's real-time emotions.

[1954] (Application example 2)

[1955] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1956] In conventional customer service work, the skills and know-how of highly productive workers are individual and cannot be easily transferred to other workers. This makes it difficult for all workers to have consistently high levels of customer service skills, leading to variations in customer satisfaction. Furthermore, it is not possible to respond appropriately to customers' emotions, which can result in a decline in the quality of service.

[1957] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1958] In this invention, the server includes means for recording talk scripts and know-how of highly productive workers, means for preprocessing the recorded talk scripts and know-how, means for training a generative artificial intelligence (AI) to learn the preprocessed data, means for evaluating and improving the performance of the AI ​​model, means for creating high-performance talk scripts and distributing them to workers, means for analyzing the voices of the workers in real time and recognizing customer emotions, means for dynamically presenting optimal talk scripts based on the customer emotions, and means for collecting feedback from the workers and further improving the AI ​​model. This allows all workers to have uniformly high levels of customer service skills and be able to respond appropriately to customers' emotions, which is expected to improve customer satisfaction.

[1959] A "highly productive worker" is a worker who has high productivity and skills in customer service work and can increase customer satisfaction.

[1960] A "talk script" refers to a series of responses and greetings that a worker uses when serving a customer.

[1961] "Know-how" refers to specific practical methods based on the skills, knowledge, and experience of highly productive workers.

[1962] "Means of recording" refers to a method in which highly productive workers record conversations and know-how while serving customers on a terminal and save it as data that can be analyzed later.

[1963] "Preprocessing means" refers to the technical process of converting recorded data into a format that is easier to analyze and removing noise and unnecessary information.

[1964] "Generative AI" is a form of AI that learns from large amounts of data, recognizes patterns, and generates new information and responses.

[1965] "Training" is the process of feeding preprocessed data into a generative artificial intelligence model and training it based on human supervision.

[1966] "Means for evaluating and improving performance" refers to methods for evaluating whether a generative artificial intelligence model is functioning properly and providing feedback and re-training to improve performance as necessary.

[1967] The "distribution means" is a method for providing the generated high-performance talk script to all workers so that it can be used uniformly.

[1968] "Means for analyzing voice in real time" refers to technology that analyzes the voice of workers on the spot and instantly understands the content of conversations with customers.

[1969] "Means for recognizing emotions" refers to technology that analyzes and identifies a customer's emotional state from real-time voice data and facial expressions.

[1970] The "dynamic presentation means" is a method of instantly changing an appropriate talk script according to the recognized customer emotions and providing it to the worker.

[1971] "Means for collecting feedback" refers to technology that records evaluations and opinions from workers after use and uses them to improve generative artificial intelligence.

[1972] The system embodying the present invention aims to standardize the customer service skills of highly productive workers so that all workers can provide high-level customer service. This system is realized by combining multiple means.

[1973] First, the user records the conversations and interactions they have with customers using a smartphone or smart glasses, and the recorded data is uploaded to a server.

[1974] The server then preprocesses the uploaded data: the audio data is converted to text using Google Cloud Speech-to-Text, noise and unnecessary information are removed, and the text data is formatted into a unified format.

[1975] After preprocessing, the data is trained by a generative artificial intelligence model (e.g., OpenAI GPT-4). The server then uses this data to execute a learning process to generate highly productive worker skills and effective conversation scripts. This learning process also takes into account customer emotions, enabling a more human-like response.

[1976] The generated generative artificial intelligence model is then evaluated for its performance. The server evaluates the model's performance using a separate validation dataset and improves the model as necessary based on the results.

[1977] After the model has been fully optimized, this high-performance conversation script is distributed to the worker. The worker then uses the distributed script to serve customers. During this process, the worker's voice is analyzed in real time, and the customer's emotions are recognized using the Microsoft Azure Emotion API. Based on this information, the optimal conversation script is dynamically presented. For example, if a customer expresses dissatisfaction, the script displayed reads, "I'm sorry. I'd like to know more about this. What is the problem?"

[1978] After the training session, the workers record their feedback in a dedicated interface and send it to the server, which analyzes the collected feedback and retrains the generative AI model as needed to improve its quality.

[1979] These processes will ensure that all staff have a uniformly high level of customer service skills, enabling them to respond appropriately to customers' emotions, and are expected to improve customer satisfaction.

[1980] Specific examples

[1981] Prompt sentence for generative AI model

[1982] "Generate a conversation script to be used when a customer is unhappy. For example, include phrases such as, 'Sorry, customer. What is the problem?'"

[1983] In this way, with the right prompts, the generative artificial intelligence model can generate high-quality conversation scripts, ensuring that all workers deliver a consistent, high-quality experience to customers.

[1984] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1985] Step 1:

[1986] Data collection

[1987] Input: Highly productive workers use smartphones or smart glasses to record conversations and responses while serving customers.

[1988] Processing: The device temporarily stores the recorded data.

[1989] Output: Saved audio file.

[1990] Specific operation: The user activates the recording function of the smart device and starts recording.

[1991] Step 2:

[1992] Data upload

[1993] Input: The audio file saved in step 1.

[1994] Processing: The device uploads the audio file to the server.

[1995] Output: Audio data uploaded to the server.

[1996] Specific operation: After the user has finished recording, they use a dedicated application to send the data to the server.

[1997] Step 3:

[1998] Data Preprocessing

[1999] Input: Audio data uploaded to the server.

[2000] Processing: The server converts the audio data into text using Google Cloud Speech-to-Text, removing noise and unnecessary information.

[2001] Output: Preprocessed text data.

[2002] Specific operation: The server uses a speech recognition engine to analyze the voice data and converts the results into a standard format.

[2003] Step 4:

[2004] Learning with generative artificial intelligence models

[2005] Input: Preprocessed text data.

[2006] Processing: The server uses the OpenAI GPT-4 generative artificial intelligence model to learn useful talking scripts and know-how from the pre-processed data.

[2007] Output: A trained generative artificial intelligence model.

[2008] Specific operation: The server trains repeatedly for the specified number of epochs and generates the optimal model.

[2009] Step 5:

[2010] Model evaluation

[2011] Input: A trained generative artificial intelligence model and a separate validation dataset.

[2012] Processing: The server evaluates the performance of the generative artificial intelligence model using a validation dataset.

[2013] Output: Evaluation results and necessary improvements.

[2014] Specific operation: The server automatically runs the evaluation tool and generates performance metrics.

[2015] Step 6:

[2016] Talk script generation and distribution

[2017] Input: Trained model and evaluation results.

[2018] Processing: The server generates a high-performance talk script and distributes it to the worker.

[2019] Output: Talk script distributed to the worker.

[2020] Specific operation: The server automatically distributes the talk script via text message or application.

[2021] Step 7:

[2022] Real-time analytics and emotion recognition

[2023] Input: Real-time voice data of the worker.

[2024] Processing: The device uses the Microsoft Azure Emotion API to recognize customer emotions in real time.

[2025] Output: Emotion recognition result.

[2026] Specific operation: The device transmits voice data to the analysis server and acquires emotion data in real time.

[2027] Step 8:

[2028] Dynamic ...

Claims

1. A means for recording talk scripts and know-how of highly productive workers; A means for pre-processing the recorded talk script and know-how; A means for training a generative artificial intelligence on the preprocessed data; A means for evaluating and improving the performance of generative artificial intelligence models; A means to create and distribute high-performance talk scripts to workers, means for collecting feedback from operators to further improve the generative artificial intelligence model; A system including:

2. 10. The system of claim 1, further comprising means for standardizing the pre-processed data into a format before training the generative artificial intelligence model.

3. 10. The system of claim 1, further comprising means for an operator to record feedback after said training session.

Images