System

Abstract

To provide a system.SOLUTION: A system includes a natural language processing model that has learned talk of a high productivity crew and talk of a low productivity crew, and includes: means by which the natural language processing model analyzes talk of a crew and determines whether the talk of the crew is good or bad; means by which the natural language processing model corrects the talk of the crew; means for collecting talk data of the crew and transmitting it to a server; means for analyzing feeling of the user using a feeling engine capable of recognizing the feeling of the user who talks with the crew; means for integrating an enhancement point of talk skill of the crew, an improvement point, and a feeling recognition result of the user and generating feedback; and means for providing the crew with the generated feedback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There are differences in productivity, opening rates, etc. between crews working at shops, mass retailers, call centers, and agencies, and the cause of this is the issue of inconsistencies in the knowledge and conversation of crews. [Means for solving the problem]

[0005] To solve this problem, we have introduced a natural language processing model that learns the talk of highly productive and less productive crew members, analyzes the crew's talk, and determines what is good and what is bad. Furthermore, the natural language processing model corrects the crew's talk and provides feedback to the crew, thereby promoting the improvement of their skills. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, the coded communication I / F (Interface) refers to a communication processor and

[0013] The communication I / F is an interface that includes a network interface (NIC) and an antenna. The communication I / F controls communication between multiple computers. Examples of communication standards that can be applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0014] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0015] [First embodiment]

[0016] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0026] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0027] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0028] "Example 1"

[0029] In one embodiment of the present invention, the natural language processing model learns the conversations of highly productive and less productive crew members. Specifically, data such as phrases, expressions, and conversation flow used by crew members during conversations with customers is collected and used as training data. This training data serves as a criterion for judging the quality of a crew member's conversation.

[0030] "Example 2"

[0031] Furthermore, the natural language processing model corrects the crew's conversations. Specifically, when the crew interacts with a customer, the content of the conversation is analyzed in real time and corrections and advice are provided as needed. For example, if the crew uses an inappropriate expression while speaking to a customer, the natural language processing model detects the expression and provides instructions to the crew to correct it to an appropriate expression.

[0032] "Example 3"

[0033] The natural language processing model also provides feedback to help crew members improve their skills. Specifically, based on the results of the corrections to the crew's conversations, it clarifies areas for improvement and strengthens the crew's conversation skills, and provides this feedback to the crew. This allows the crew to objectively understand their own conversation skills and take concrete steps to improve them.

[0034] The processing flow of each embodiment will be described below.

[0035] "Example 1"

[0036] Step 1: The natural language processing model learns the conversations of high-productivity and low-productivity crew members. Specifically, it captures data such as phrases, expressions, and conversation flow used by crew members during conversations with customers.

[0037] Step 2: The captured data is used as training data to train a natural language processing model. Through this training, the model learns the criteria for judging whether a crew member's talk is good or bad.

[0038] "Example 2"

[0039] Step 1: The natural language processing model analyzes crew conversations in real time. Specifically, it captures the conversations that crew members make with customers in real time.

[0040] Step 2: Analyze the captured conversation and provide corrections or advice as needed. For example, if a crew member uses inappropriate language when speaking to a customer, the natural language processing model will detect that expression.

[0041] Step 3: Provide the crew with instructions to correct the detected inappropriate expressions into appropriate expressions. This allows the crew to improve their own talk in real time. "Example 3"

[0042] Step 1: Based on the results of the crew's talk corrections, the natural language processing model identifies areas for strengthening and improving the crew's talking skills.

[0043] Step 2: Provide feedback to the crew on the identified areas for improvement. This allows the crew to objectively understand their own speaking skills and take concrete steps to improve.

[0044] Example 1

[0045] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] In conventional crew dialogue evaluation systems, the criteria for judging the quality of dialogue were vague, making it difficult to effectively promote crew skill improvement. Furthermore, the steps of voice data text conversion, data preprocessing, feature extraction, model learning, evaluation, and feedback were not integrated, making efficient system operation difficult.

[0047] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0048] In this invention, the server includes means for converting voice data into text data, means for preprocessing the text data, means for extracting features from the preprocessed data, means for training a model using the extracted features, means for evaluating the performance of the trained model, means for improving the model based on the evaluation results, means for analyzing the crew's dialogue and determining what is good and what is bad, and means for correcting the crew's dialogue. This makes it possible to accurately determine the quality of the crew's dialogue and provide effective feedback.

[0049] A "highly productive crew" is a crew that can achieve high results in interactions with customers.

[0050] A "low-productivity crew" refers to a crew member who performs poorly in interactions with customers.

[0051] "Natural language processing model" refers to an algorithm or machine learning model for understanding and processing natural language.

[0052] "Dialogue" refers to the verbal exchange between the crew and the customer.

[0053] "Audio Data" refers to audio files that record the conversation between the crew and the customer.

[0054] "Text data" refers to data obtained by converting voice data into character information.

[0055] "Preprocessing" refers to the process of removing unnecessary information from text data and converting it into a format that is easy to analyze.

[0056] "Feature extraction" refers to the process of extracting useful information and patterns from text data.

[0057] "Model learning" refers to the process of training a machine learning model using the extracted features.

[0058] "Evaluation" refers to the process of measuring the performance of a trained model and confirming its accuracy and effectiveness.

[0059] "Feedback" refers to information provided to the crew members to promote skill improvement through evaluation results of the model and corrections to the dialogue.

[0060] The present invention is a system including a natural language processing model for learning the dialogue between a high-productivity crew and a low-productivity crew. A specific embodiment of this system will be described below.

[0061] System Overview

[0062] The user generates a program that uses a natural language processing model to learn the conversations between high-productivity and low-productivity crew members. This program takes in data such as phrases, expressions, and the flow of conversations used by crew members when interacting with customers, and uses this data as learning data.

[0063] Hardware and software used

[0064] Server: Collects data, preprocesses it, extracts features, trains it, evaluates it, and provides feedback.

[0065] Speech recognition software: converts voice data into text data (e.g., Google® Speech-to-Text API).

[0066] Natural language processing libraries: Preprocess text data (e.g., NLTK, spaCy).

[0067] Machine learning libraries: for feature extraction and model training (e.g., scikit-learn, TENSORFLOW®, PyTorch).

[0068] Data processing and calculation

[0069] 1. Data Collection:

[0070] The server records the phrases and expressions used by the crew members while interacting with customers, as well as the flow of the conversation, and uses voice recognition software to convert the voice data into text data.

[0071] 2. Data Preprocessing:

[0072] The server preprocesses the collected text data, specifically by removing noise, tokenizing, and removing stop words.

[0073] 3. Feature Extraction:

[0074] The server extracts features from the preprocessed data, such as frequent phrases, sentiment analysis, and dialogue flow.

[0075] 4. Model training:

[0076] The server uses the extracted features to train a natural language processing model. The model is trained using the dialogue data of the high-productivity crew and the low-productivity crew as labeled data.

[0077] 5. Rating and Feedback:

[0078] The server evaluates the performance of the trained model using metrics such as precision, recall, and F1 score, improves the model based on the evaluation results, and provides feedback to the crew.

[0079] Specific examples

[0080] Example dialogue from a high-productivity crew: "Customers, this product is very popular. This feature is especially popular."

[0081] Example dialogue from a low-productivity crew: "Well, this product is okay. Nothing special."

[0082] Prompt Sentence Examples

[0083] "Generate a program to train a natural language processing model using the dialogue data of high-productivity and low-productivity crews. Specifically, the process will include the steps of collection, preprocessing, feature extraction, model training, and evaluation."

[0084] In this way, users can build a system that uses a natural language processing model to learn criteria for judging whether a crew member's dialogue is good or bad.

[0085] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0086] Step 1: Data collection

[0087] Input: Voice data of conversation between crew and customer

[0088] What it does: The server records the phrases and expressions used by the crew members while interacting with the customer, as well as the flow of the conversation. It then converts the voice data into text using speech recognition software (e.g., Google Speech-to-Text API).

[0089] Output: Text data

[0090] Step 2: Data Preprocessing

[0091] Input: Collected text data

[0092] Specific operation: The server preprocesses the collected text data, specifically removing noise, tokenizing, and removing stop words. This process uses natural language processing libraries (e.g., NLTK, spaCy).

[0093] Output: Preprocessed text data

[0094] Step 3: Feature extraction

[0095] Input: Preprocessed text data

[0096] Specific operation: The server extracts features from the preprocessed data, such as frequently used phrases, sentiment analysis, and dialogue flow. This process uses machine learning libraries (e.g., scikit-learn, TensorFlow).

[0097] Output: Extracted feature data

[0098] Step 4: Model training

[0099] Input: Extracted feature data and label data (interaction data of high-productivity crew and low-productivity crew)

[0100] Specific operation: The server uses the extracted features to train a natural language processing model. The conversation data of the high-productivity crew and the low-productivity crew are used as labeled data to train the model. This process uses a deep learning framework (e.g., PyTorch, TensorFlow).

[0101] Output: Trained model

[0102] Step 5: Rating and feedback

[0103] Input: trained model and test data

[0104] Specific operation: The server evaluates the performance of the trained model using metrics such as precision, recall, and F1 score, improves the model based on the evaluation results, and provides feedback to the crew.

[0105] Output: Evaluation results and feedback information

[0106] In this way, users can build a system that uses a natural language processing model to learn criteria for judging whether a crew member's dialogue is good or bad.

[0107] (Application example 1)

[0108] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] In conventional customer service work, there was a problem in which variations in crew conversation skills affected customer satisfaction. In particular, the conversations of low-productivity crew were often ineffective in communicating with customers, which led to lower sales and customer satisfaction. Another issue was the lack of feedback and training for crew to improve their conversation skills. To solve these problems, a system was needed that would help crew use appropriate phrases and expressions in real time and promote the improvement of their conversation skills.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0111] In this invention, the server includes a natural language processing model for learning the talk of high-productivity crew members and low-productivity crew members, means for the natural language processing model to analyze the crew members' talk and determine what is good and what is bad, means for the natural language processing model to correct the crew members' talk, means for suggesting appropriate phrases and expressions in real time when the crew members interact with customers, means for evaluating the crew members' talk and providing feedback, and means for the crew members to use during training to acquire talking skills, thereby enabling the crew members to communicate effectively in interactions with customers, improving customer satisfaction and increasing sales.

[0112] A "highly productive crew" is an employee who has excellent communication skills when interacting with customers and is highly efficient at work.

[0113] "Low-productivity crew" refers to employees who lack communication skills when interacting with customers and have low work efficiency.

[0114] A "natural language processing model" is a model that uses algorithms and techniques to understand and generate human language.

[0115] "Talk" refers to the words, phrases, and expressions used by crew members when interacting with customers.

[0116] "Analysis" refers to the process of capturing crew talk as data and evaluating its content.

[0117] "Judgment" refers to the process of evaluating the quality of a crew member's talk based on evaluation criteria.

[0118] "Correction" is the process of correcting the crew's talk and changing it to more appropriate expressions.

[0119] "Real-time" means responding instantly to the moment the crew is interacting with the customer.

[0120] "Phrases" are short words or expressions that crew members use when interacting with customers.

[0121] "Evaluation" is the process of measuring the effectiveness and appropriateness of the crew's talk based on its content.

[0122] "Feedback" is the process of communicating the results of the crew's talk evaluation and pointing out areas for improvement and good points.

[0123] "Training" refers to the learning and practice that crew members undertake to improve their skills.

[0124] "Skills" refer to the techniques and abilities that crew members use in interacting with customers.

[0125] As an embodiment of the present invention, the following system can be constructed.

[0126] System Overview

[0127] The system suggests appropriate phrases and expressions in real time when the crew interacts with customers, and provides evaluations and feedback on the conversation. The system consists of a server and a terminal including smart glasses.

[0128] Hardware and software used

[0129] Hardware:

[0130] Smart glasses: equipped with a display and microphone, worn by the crew.

[0131] Server: A cloud server to host the natural language processing model and process data in real time.

[0132] software:

[0133] spaCy: A natural language processing library.

[0134] Transformers: Hugging Face's Transformers model library.

[0135] GPT-2: A generative AI model.

[0136] Data processing and calculation

[0137] 1. Talk Analysis:

[0138] Voice data from crew members interacting with customers is collected by the microphone in the smart glasses and converted into text data.

[0139] The server analyzes the collected text data using spaCy and evaluates the content of the talk.

[0140] 2. Real-time support:

[0141] The crew's input text is fed into the GPT-2 model to generate appropriate phrases and expressions.

[0142] The generated phrases and expressions are displayed in real time on the smart glasses' display.

[0143] 3. Talk evaluation and feedback:

[0144] The server evaluates the crew's talk and calculates a score.

[0145] The evaluation results are provided as feedback to the crew and used in training mode.

[0146] Specific examples

[0147] When the crew member asks, "Hello, customer. What products are you looking for today?", the smart glasses' display will show, "We have a new collection in stock. Please take a look."

[0148] Prompt Sentence Examples

[0149] Customer: Hello, do you have any recommendations for me today?

[0150] Crew: Hello, customer. What product are you looking for today?

[0151] In this way, the system supports crew conversations in real time, enabling smoother dialogue with customers, and can also encourage crew skill improvement through conversation evaluation and feedback.

[0152] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0153] Step 1:

[0154] The crew member begins a conversation with the customer.

[0155] Input: Crew voice.

[0156] Specific operation: The crew puts on the smart glasses and starts talking to the customer. The microphone in the smart glasses collects the crew's voice.

[0157] Step 2:

[0158] The terminal converts the voice data into text data.

[0159] Input: Crew voice data.

[0160] Data processing: Converting voice data into text using speech recognition software.

[0161] Output: Crew text data.

[0162] Specific operation: The built-in microphone of the smart glasses collects voice data and converts it into text data in real time.

[0163] Step 3:

[0164] The server analyzes the text data and generates appropriate phrases and expressions.

[0165] Input: Crew text data.

[0166] Data Computation: Analyzes text data using a natural language processing model (spaCy) and generates appropriate phrases and expressions using a generative AI model (GPT-2).

[0167] Output: Suggested phrases or expressions.

[0168] How it works: The server receives the text data, analyzes it, and then uses a generative AI model to generate appropriate phrases and expressions.

[0169] Step 4:

[0170] The device displays the generated phrases and expressions to the crew in real time.

[0171] Input: A suggested phrase or expression.

[0172] Output: A phrase or expression that will be displayed on the smart glasses display.

[0173] What it does: The generated phrases and expressions are displayed in real time on the smart glasses display, allowing the crew member to refer to them as they continue their conversation with the customer.

[0174] Step 5:

[0175] The server rates the crew's talk and provides feedback.

[0176] Input: Crew text data.

[0177] Data computation: Using natural language processing models to evaluate the content of the talk and calculate a score.

[0178] Output: Evaluation results as feedback.

[0179] What it does: The server evaluates the crew's talk, calculates a score, and generates feedback.

[0180] Step 6:

[0181] Crew members receive feedback and improve their skills in training mode.

[0182] Input: Evaluation results as feedback.

[0183] Output: Crew skill improvement.

[0184] What it does: Crew members receive feedback and practice to improve their speaking skills in training mode.

[0185] Example 2

[0186] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0187] Conventional dialogue support systems have the problem that it is difficult for workers to receive appropriate feedback on the spot when they use inappropriate expressions during a conversation with a customer. Additionally, there is a lack of specific advice to improve the quality of the conversation, which means that workers are unable to sufficiently improve their skills.

[0188] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0189] In this invention, the server includes means for analyzing the worker's dialogue in real time and determining what is good and what is bad, means for correcting the worker's dialogue, means for detecting inappropriate expressions and providing the worker with instructions to correct them to appropriate expressions, means for transmitting the generated advice to the worker's terminal, and means for the terminal to display the advice to the worker. This enables the worker to receive appropriate feedback in real time during a dialogue with a customer, and by providing specific advice to improve the quality of the dialogue, the worker's skills can be improved.

[0190] A "highly productive worker" refers to a worker who achieves high efficiency and results in their work.

[0191] "Low productivity workers" refer to workers who are low in efficiency and produce low results in their work.

[0192] A "natural language processing model" refers to an algorithm or machine learning model that understands and generates human language.

[0193] "Dialogue" refers to communication between a worker and a customer.

[0194] "Real-time analysis" refers to analyzing data instantly while a conversation is taking place.

[0195] "Inappropriate language" refers to words or phrases that may be offensive or misleading to customers.

[0196] "Appropriate expressions" refer to words and phrases that leave a favorable impression on customers and convey accurate information.

[0197] "Advice" refers to specific instructions or suggestions for the worker to improve the interaction.

[0198] "Terminal" refers to a computer or mobile device used by a worker.

[0199] "Feedback" refers to evaluations and information on areas for improvement provided to workers.

[0200] "Skill improvement" refers to improving workers' work abilities and communication skills.

[0201] The present invention provides a system for providing appropriate feedback in real time when a worker is interacting with a customer. The system includes a server, a terminal, and a natural language processing model.

[0202] Hardware and software used

[0203] Hardware: Servers, terminals (devices used by workers)

[0204] Software: Natural language processing models (e.g., GPT-4®), real-time data analysis tools (e.g., Apache® Kafka)

[0205] System Operation

[0206] 1. The user (worker) initiates a conversation with the customer. The worker uses a device (e.g., a tablet or PC) to communicate with the customer. The conversation is recorded on the device via voice input or text input.

[0207] 2. The device sends the conversation content to the server in real time using the WebSocket or HTTP protocol.

[0208] 3. The server inputs the received dialogue into the natural language processing model. The input data is in text format, and is preprocessed as necessary.

[0209] 4. The natural language processing model analyzes the dialogue and identifies inappropriate expressions and areas for improvement, such as "This product may not be suitable for you."

[0210] 5. The server receives the analysis results from the natural language processing model and generates appropriate expressions and advice, such as "You might want to compare this product with other options."

[0211] 6. The server sends the generated advice to the device using the WebSocket or HTTP protocol.

[0212] 7. The terminal displays the received advice to the user (worker) in the form of a pop-up or notification.

[0213] 8. The user (worker) modifies the dialogue based on the displayed advice. For example, they might say, "You might want to compare this product with other options."

[0214] Specific examples

[0215] Scenario: A worker says to a customer, "This product may not be right for you."

[0216] Analysis result of natural language processing model: "Inappropriate expressions were detected. These may create a negative impression on customers."

[0217] Generated advice: "You might want to compare this product with other options."

[0218] Prompt Sentence Examples

[0219] Input prompt: "When a worker says to a customer, 'This product may not be right for you,' please provide advice on how to revise it to an appropriate phrase."

[0220] Sample output: "You might want to compare this product with other options."

[0221] The system provides real-time support to help workers smoothly advance their interactions with customers.

[0222] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0223] Step 1:

[0224] The user (worker) starts a dialogue with the customer.

[0225] Input: The worker uses the terminal to interact with the customer.

[0226] Specific operation: The worker records the conversation content on the terminal through voice input or text input.

[0227] Output: The conversation is logged to the terminal.

[0228] Step 2:

[0229] The terminal transmits the contents of the conversation to the server in real time.

[0230] Input: Dialogue recorded on device.

[0231] Specific operation: The device captures the conversation content in real time and sends it to the server using WebSocket or HTTP protocol.

[0232] Output: The dialogue is sent to the server.

[0233] Step 3:

[0234] The server inputs the received dialogue content into a natural language processing model.

[0235] Input: The conversation sent to the server.

[0236] Specific operation: The server inputs the received dialogue content in text format into the natural language processing model, and performs preprocessing as necessary.

[0237] Output: The dialogue input to the natural language processing model.

[0238] Step 4:

[0239] A natural language processing model analyzes the dialogue and detects inappropriate expressions and areas for improvement.

[0240] Input: The dialogue that is fed into the natural language processing model.

[0241] How it works: The natural language processing model analyzes the conversation and identifies inappropriate expressions and areas for improvement, such as "This product may not be suitable for you."

[0242] Output: Detected inappropriate expressions and areas for improvement.

[0243] Step 5:

[0244] The server receives the analysis results from the natural language processing model and generates appropriate expressions and advice.

[0245] Input: Analysis results from a natural language processing model.

[0246] Specific operation: The server generates appropriate expressions and advice based on the analysis results. For example, it generates advice such as "You may want to compare this product with other options."

[0247] Output: The generated advice.

[0248] Step 6:

[0249] The server sends the generated advice to the terminal.

[0250] Input: The generated advice.

[0251] Specific operation: The server sends the generated advice to the device using WebSocket or HTTP protocol.

[0252] Output: Advice is sent to the terminal.

[0253] Step 7:

[0254] The terminal displays advice to the crew.

[0255] Input: Advice sent to the terminal.

[0256] Specific operation: The terminal displays the received advice to the worker in the form of a pop-up or notification.

[0257] Output: The advice displayed to the worker.

[0258] Step 8:

[0259] The user (operator) corrects the dialogue based on the displayed advice.

[0260] Input: The advice shown.

[0261] Specific behavior: The worker corrects the dialogue based on the displayed advice, for example, saying, "You might want to compare this product with other options."

[0262] Output: The corrected dialogue.

[0263] (Application example 2)

[0264] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0265] In traditional customer service work in brick-and-mortar stores, the content of the conversation between workers has a significant impact on customer satisfaction, but if a worker uses inappropriate language, it is difficult to correct it on the spot. Also, in order to improve the worker's skills, the content of the conversation must be fed back after the fact, making it difficult to improve in real time. This has led to issues such as a decrease in customer satisfaction and delays in improving worker skills.

[0266] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0267] In this invention, the server includes means for analyzing the worker's dialogue and determining what is good and what is bad, means for correcting the worker's dialogue, means for analyzing the worker's dialogue in real time and providing instructions for correcting to appropriate expressions, and means for displaying the corrected expressions on a display device. This makes it possible to correct to appropriate expressions in real time even if the worker uses inappropriate expressions during a dialogue with a customer, thereby improving customer satisfaction and worker skills at the same time.

[0268] A "highly productive worker" is a worker who achieves high efficiency and results in their work.

[0269] A "low-productivity worker" is a worker who is inefficient or produces poor results in their work.

[0270] "Dialogue" refers to verbal exchanges between a worker and a customer.

[0271] A "natural language processing model" is a model that uses artificial intelligence technology to understand, analyze, and generate human language.

[0272] "Analysis" is the process of examining the content of a conversation in detail and determining what is good and what is bad.

[0273] "Correction" refers to correcting the content of a conversation and changing it to appropriate expression.

[0274] "Real-time" refers to processing and analysis occurring at the exact moment the interaction is taking place.

[0275] "Appropriate expression" means using correct language that is not rude or misleading to customers.

[0276] "Instructions" are specific advice or orders given to workers to correct their behavior or words.

[0277] "Display device" means a device for visually displaying modified expressions or instructions.

[0278] A system for implementing the present invention includes smart glasses worn by a worker, a microphone for recognizing voice, a server for executing a natural language processing model, and a display device. A specific embodiment of this system will be described below.

[0279] First, a worker wears smart glasses and interacts with a customer through a microphone. The microphone captures the worker's voice in real time and sends the voice data to a server. The server then converts the voice data into text data using speech recognition software (e.g., the SpeechRecognition library).

[0280] The server then inputs the converted text data into a natural language processing model (e.g., the BERT model from the Transformers library) to analyze the dialogue. The natural language processing model determines whether the dialogue is appropriate and, if it contains inappropriate language, generates instructions to correct it to an appropriate expression.

[0281] The generated correction instructions are displayed in real time on the smart glasses' display device. This allows the worker to check the appropriate expression during the conversation and make immediate corrections. For example, if the worker says, "This product is cheap," the natural language processing model will determine that the expression "cheap" is negative, and the smart glasses will display a correction instruction to read, "This product is reasonably priced."

[0282] This system allows workers to learn appropriate expressions in real time while dealing with customers, thereby improving customer satisfaction and worker skills at the same time.

[0283] An example of a prompt is as follows:

[0284] "If a crew member says to a customer, 'This product is cheap,' generate instructions to correct that."

[0285] In this way, the quality of customer service provided by staff in physical stores can be improved.

[0286] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0287] Step 1:

[0288] The worker wears smart glasses and communicates with the customer through a microphone.

[0289] Input: Worker's voice

[0290] Output: Audio data

[0291] Specific action: The worker initiates a conversation with the customer, and the microphone captures their voice.

[0292] Step 2:

[0293] The audio data captured by the microphone is sent to the server.

[0294] Input: Audio data

[0295] Output: Audio data sent to the server

[0296] Specific operation: The audio data captured by the microphone is sent to the server in real time.

[0297] Step 3:

[0298] The server uses voice recognition software to convert the voice data into text data.

[0299] Input: Audio data

[0300] Output: Text data

[0301] Specific operation: The server uses the SpeechRecognition library to analyze the audio data and generate corresponding text data.

[0302] Step 4:

[0303] The server inputs text data into a natural language processing model and analyzes the content of the conversation.

[0304] Input: Text data

[0305] Output: Analysis results

[0306] Specific operation: The server analyzes the text data using the BERT model from the Transformers library and evaluates the appropriateness of the dialogue content.

[0307] Step 5:

[0308] If the server finds that an inappropriate expression is included, it generates instructions to correct it to an appropriate expression.

[0309] Input: Analysis results

[0310] Output: Correction instructions

[0311] Specific operation: Based on the analysis results of the natural language processing model, the server detects inappropriate expressions and generates instructions to correct them to appropriate expressions.

[0312] Step 6:

[0313] The server transmits the generated correction instructions to the display device of the smart glasses.

[0314] Input: Correction instructions

[0315] Output: Correction instructions displayed on the smart glasses

[0316] Specific operation: The server sends the correction instructions to the display device of the smart glasses so that the worker can check them.

[0317] Step 7:

[0318] The worker checks the correction instructions displayed on the display device of the smart glasses and corrects the content of the dialogue.

[0319] Input: Correction instructions displayed on smart glasses

[0320] Output: Corrected dialogue

[0321] Specific operations: The worker checks the correction instructions displayed on the smart glasses' display and corrects the content of the conversation with the customer to use appropriate expressions.

[0322] Example 3

[0323] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0324] Conventional systems for improving crew conversation skills have had problems such as insufficient analysis of conversation content and feedback, making it difficult for crew members to identify specific areas for improvement. Furthermore, saving conversation content and managing analysis results are often done manually, making it difficult to improve skills efficiently.

[0325] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0326] In this invention, the server includes a means for the crew to input talk content, a means for the server to receive and store the talk content, a means for the server to send the talk content to a natural language processing model, a means for the natural language processing model to analyze the talk content, a means for the server to receive and store the analysis results, a means for the server to generate feedback, and a means for the crew to receive feedback, which makes it possible to efficiently analyze the talk content of the crew and provide specific feedback.

[0327] A "highly productive crew" is a crew that can achieve high results in their work.

[0328] A "low-productivity crew" refers to a crew member who performs poorly in their work.

[0329] A "natural language processing model" refers to an algorithm or machine learning model that analyzes the content of crew conversations and extracts areas for improvement or strengthening.

[0330] "Talk content" refers to the content of conversations and comments made by crew members while on duty.

[0331] "Server" refers to a computer system that receives and stores crew chat content and manages the analysis results.

[0332] "Feedback" refers to specific advice and areas for improvement provided to the crew based on the results of analysis by the natural language processing model.

[0333] A "prompt sentence" refers to an instruction sentence used when requesting analysis from a natural language processing model.

[0334] "Analysis results" refers to information on areas for enhancement or improvement obtained after the natural language processing model analyzes the content of the talk.

[0335] "Database" refers to an information management system for storing talk content and analysis results.

[0336] "Input form" refers to the interface that crew members use to enter talk content.

[0337] This invention is a system that provides feedback to improve crew members' speaking skills. The system analyzes the content of crew members' conversations using a natural language processing model, and provides feedback by clarifying areas for improvement.

[0338] Hardware and software used

[0339] Hardware: Servers, devices (PCs and smartphones)

[0340] Software: Natural language processing models (e.g., GPT-4), database management systems (e.g., MySQL®), web application frameworks (e.g., Django)

[0341] Specific operation of the system

[0342] The user enters the content of the chat.

[0343] The user (crew member) uses the terminal to input the content of the conversation. For example, they might type "Good morning. How can I help you today?" into an input form on a web browser.

[0344] The server receives and stores the chat content.

[0345] The server receives the chat content sent from the device. The server saves the received chat content in a database (e.g., MySQL). When saving, it also records the user ID and timestamp.

[0346] The server sends the conversation content to the natural language processing model

[0347] The server sends the saved chat content to a natural language processing model (e.g., GPT-4). When sending, it generates a prompt and passes it to the model along with the chat content. For example, it generates a prompt like, "Please analyze the following chat content and provide feedback on areas for improvement and strengthening. Chat content: 'Good morning. What can we do for you today?'"

[0348] Natural language processing model analyzes the content of the conversation

[0349] The natural language processing model analyzes the content of the conversation based on the prompt received. As a result of the analysis, it extracts areas for improvement and strengthening. For example, it generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[0350] The server receives and stores the analysis results.

[0351] The server receives the analysis results from the natural language processing model and stores them in a database. When storing them, the server associates the conversation content with the corresponding analysis results.

[0352] The server generates feedback

[0353] The server generates feedback based on the analysis results. The feedback is provided in a format that is easy for the user to understand, and includes specific advice such as "next time, speak in a softer tone" or "use specific friendly words."

[0354] User receives feedback

[0355] The user can then use the device to review the generated feedback, which is displayed in a web browser. Based on the feedback, the user can take specific actions to improve their speaking skills.

[0356] Examples of specific examples and prompts

[0357] Examples:

[0358] The crew enters "how to greet customers."

[0359] The natural language processing model generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[0360] The crew will take steps to improve, such as "next time, speak in a softer tone" or "use specific, friendly words."

[0361] Example prompt sentence:

[0362] Please analyze the following conversation and provide feedback on areas for improvement. Conversation: 'Good morning. How can I help you today?'

[0363] In this way, the system supports the crew in improving their talking skills. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[0364] Step 1:

[0365] The user enters the content of the chat.

[0366] The user (crew member) uses the terminal to input the content of the conversation. For example, they enter "Good morning. How can I help you today?" into the input form. The entered content of the conversation is sent to the system.

[0367] Input: The chat content entered by the user

[0368] Output: The message is sent to the server.

[0369] Step 2:

[0370] The server receives and stores the chat content.

[0371] The server receives the chat content sent from the device and saves it in a database (e.g., MySQL). When saving, it also records the user ID and timestamp.

[0372] Input: Chat content sent from the device

[0373] Output: The chat content is saved in the database.

[0374] Step 3:

[0375] The server sends the conversation content to the natural language processing model

[0376] The server sends the saved chat content to a natural language processing model (e.g., GPT-4). When sending, it generates a prompt and passes it to the model along with the chat content. For example, it generates a prompt like, "Please analyze the following chat content and provide feedback on areas for improvement and strengthening. Chat content: 'Good morning. What can we do for you today?'"

[0377] Input: Chat content stored in the database

[0378] Output: The conversation along with the prompt is sent to the natural language processing model.

[0379] Step 4:

[0380] Natural language processing model analyzes the content of the conversation

[0381] The natural language processing model analyzes the content of the conversation based on the prompt received. As a result of the analysis, it extracts areas for improvement and strengthening. For example, it generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[0382] Input: Prompt and talk content

[0383] Output: Analysis results (enhancements and improvements)

[0384] Step 5:

[0385] The server receives and stores the analysis results.

[0386] The server receives the analysis results from the natural language processing model and stores them in a database. When storing them, the server associates the conversation content with the corresponding analysis results.

[0387] Input: Analysis results from a natural language processing model

[0388] Output: Analysis results are saved in a database

[0389] Step 6:

[0390] The server generates feedback

[0391] The server generates feedback based on the analysis results. The feedback is provided in a format that is easy for the user to understand, and includes specific advice such as "next time, speak in a softer tone" or "use specific friendly words."

[0392] Input: Analysis results stored in a database

[0393] Output: Feedback is generated

[0394] Step 7:

[0395] User receives feedback

[0396] The user can then use the device to review the generated feedback, which is displayed in a web browser. Based on the feedback, the user can take specific actions to improve their speaking skills.

[0397] Input: Feedback from the server

[0398] Output: User receives and confirms feedback

[0399] (Application example 3)

[0400] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] Conventional systems for improving crew conversation skills have difficulty analyzing crew conversation content in real time and providing immediate feedback. Furthermore, the feedback is often not specific, making it unclear how crew members should improve. This leads to delays in crew skill improvement.

[0402] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0403] In this invention, the server includes a natural language processing model for learning the conversations of high-productivity workers and low-productivity workers, means for the natural language processing model to analyze the workers' conversations and determine what is good and what is bad, means for the natural language processing model to correct the workers' conversations, means for the natural language processing model to analyze the workers' conversations in real time and provide feedback on areas for improvement and strengthening, and means including a display device for displaying the feedback. This allows workers to receive specific feedback in real time, enabling them to improve their skills quickly and effectively.

[0404] A "highly productive worker" is a worker who achieves high efficiency and results in their work.

[0405] A "low-productivity worker" is a worker who is inefficient or produces poor results in their work.

[0406] A "natural language processing model" is a type of artificial intelligence technology for understanding and analyzing human language.

[0407] "Worker conversation" refers to verbal communication that occurs during work.

[0408] "Analysis" is the act of examining data or information in detail to find meaning and patterns.

[0409] "Judgment" is the act of deciding whether something is good or bad or appropriate based on information.

[0410] "Correction" is the act of correcting and improving errors in writing or conversation.

[0411] "Real-time analysis" means processing data instantly and obtaining results immediately.

[0412] "Areas of improvement" are points that can be used to further improve current skills and performance.

[0413] "Areas for improvement" are points in your current skills or performance that need to be corrected or improved.

[0414] "Feedback" is the act of providing evaluation or comment on behavior or results.

[0415] A "display device" is hardware for visually displaying information.

[0416] A system for implementing this invention is configured as follows: The server includes a natural language processing model for learning conversations between high-productivity workers and low-productivity workers. The natural language processing model analyzes the workers' conversations and determines what is good and what is bad. The natural language processing model also corrects the workers' conversations, analyzes them in real time, and provides feedback on areas for improvement and strengthening. The feedback is provided to the workers via a display device.

[0417] Hardware and software used

[0418] Hardware: Smartphones (e.g., iPhone (registered trademark), Android (registered trademark) devices), smart glasses (e.g., Google Glass (registered trademark))

[0419] Software: Natural language processing models (e.g., OpenAI® GPT-4), speech recognition software (e.g., Google Speech-to-Text, IBM Watson® Speech to Text), feedback systems (e.g., custom feedback engines)

[0420] Data processing and calculation

[0421] 1. Voice recognition: The microphone on the device (smartphone or smart glasses) is used to record the worker's speech, which is then converted into text using voice recognition software.

[0422] 2. Natural language processing: The server inputs the converted text into a natural language processing model to analyze the conversation.

[0423] 3. Feedback generation: Based on the analysis results, the server generates enhancements and improvements in a feedback system.

[0424] 4. Real-time feedback: Display feedback on the device display.

[0425] Specific examples

[0426] For example, consider a situation where a staff member in a physical store is explaining a new product to a customer. If the staff member says, "This product is very good. Please try it," the natural language processing model will analyze this conversation and display feedback such as, "It would be good to explain the product's features in more detail. For example, it would be effective to give a specific example such as, 'This product uses the latest technology and is 30% more efficient than previous products.'"

[0427] Prompt Sentence Examples

[0428] Analyze the conversation of a staff member explaining a new product to a customer and generate feedback in the following format:

[0429] Conversation: [Staff conversation]

[0430] Feedback: [Improvements and improvements]

[0431] example:

[0432] Conversation: "This product is great. You should definitely try it."

[0433] Feedback: "It would be helpful to provide a more detailed description of the product's features. For example, you could say, 'This product uses cutting-edge technology and is 30% more efficient than conventional products.'"

[0434] In this way, workers receive real-time, specific feedback, enabling them to improve their skills quickly and effectively.

[0435] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0436] Step 1:

[0437] Voice Recognition

[0438] The microphone on the device (smartphone or smart glasses) is used to record the worker's conversation. The recorded voice data is input into voice recognition software (e.g., Google Speech-to-Text, IBM Watson Speech to Text) and converted into text data.

[0439] Input: Audio data

[0440] Output: Text data

[0441] Step 2:

[0442] Natural Language Processing

[0443] The server inputs the text data obtained in step 1 into a natural language processing model (e.g., OpenAI GPT-4) to analyze the conversation. The analysis extracts the content, tone, grammar, and specific patterns and phrases of the conversation.

[0444] Input: Text data

[0445] Output: Analysis results (evaluation of conversation content, specific patterns and phrases)

[0446] Step 3:

[0447] Feedback Generation

[0448] The server generates reinforcement and improvement points in the feedback system based on the analysis results obtained in step 2. The generated feedback includes specific advice and improvement measures.

[0449] Input: Analysis results

[0450] Output: Feedback data (enhancements and improvements)

[0451] Step 4:

[0452] Real-time feedback display

[0453] The terminal displays the feedback data generated in step 3 on a display device (such as a smartphone screen or smart glasses display). The worker receives the feedback in real time and can immediately implement improvements.

[0454] Input: Feedback data

[0455] Output: The displayed feedback

[0456] Specific operation example

[0457] For example, when a store staff member is explaining a new product to a customer, the process is as follows:

[0458] 1. Step 1: When the staff member says, "This product is great. Please try it," the voice is recorded by the microphone in the smart glasses and converted into text by speech recognition software.

[0459] 2. Step 2: The server inputs the converted text, "This product is very good. Please try it.", into a natural language processing model and analyzes the conversation. The analysis results point out the lack of specificity and the use of general phrases.

[0460] 3. Step 3: Based on the analysis results, the server generates feedback such as, "It would be better to explain the product's features in more detail. For example, it would be effective to provide a specific example such as, 'This product uses the latest technology and is 30% more efficient than conventional products.'"

[0461] 4. Step 4: The generated feedback is displayed on the smart glasses display, allowing staff to receive specific advice in real time.

[0462] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0463] "Example 1"

[0464] One embodiment of the present invention provides a system that uses a natural language processing model incorporating an emotion engine. This system recognizes the user's emotions from the crew's conversation and suggests appropriate responses to the crew based on those emotions. For example, if the user uses an expression that indicates anger or frustration, the emotion engine detects this and instructs the crew to respond to the user's emotions, such as by apologizing or proposing a solution.

[0465] "Example 2"

[0466] The emotion engine also extracts patterns and phrases from the crew's conversations that reflect the user's emotions. This makes it possible to grasp changes in the user's emotions in real time and suggest appropriate responses to the crew. For example, if the user uses expressions that show joy or satisfaction, the emotion engine will detect this and instruct the crew on how to maintain the user's positive emotions, such as expressing gratitude to the crew.

[0467] "Example 3"

[0468] Furthermore, the emotion engine provides feedback to the crew member on the results of corrections to their chat and the user's emotional recognition. This allows the crew member to understand how their chat affected the user's emotions and improve their skills based on the results. For example, if the crew member's proposed solution resolves the user's dissatisfaction and the user's emotions change to joy, the crew member can confirm through this feedback that their response was appropriate.

[0469] The processing flow of each embodiment will be described below.

[0470] "Example 1"

[0471] Step 1: The crew member initiates a conversation with the user.

[0472] Step 2: The natural language processing model analyzes the crew's talk, and the emotion engine recognizes the user's emotions.

[0473] Step 3: The system suggests a response to the crew based on the user's emotions recognized by the emotion engine.

[0474] "Example 2"

[0475] Step 1: The crew member initiates a conversation with the user.

[0476] Step 2: The emotion engine extracts patterns and phrases from the crew's conversation that reflect the user's emotions.

[0477] Step 3: Based on the patterns and phrases extracted by the emotion engine, the system suggests responses to the crew.

[0478] "Example 3"

[0479] Step 1: The crew member initiates a conversation with the user.

[0480] Step 2: The natural language processing model analyzes the crew's talk, and the emotion engine recognizes the user's emotions.

[0481] Step 3: The system provides feedback to the crew member on the results of corrections to their talk and the recognition of the user's emotions.

[0482] Step 4: Crew members use the feedback to improve their skills.

[0483] Example 1

[0484] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0485] Conventional crew conversation analysis systems lack the ability to not only judge the quality of a crew conversation, but also recognize the user's emotions in real time and suggest appropriate responses. This can lead to the crew not being able to respond appropriately to the user's emotions, which can result in a decline in customer satisfaction.

[0486] The identification processing by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a natural language processing model for learning the talk of high-productivity crew members and low-productivity crew members, means for the natural language processing model to analyze the crew members' talk and determine what is good and what is bad, means for the natural language processing model to correct the crew members' talk, an emotion engine for recognizing the user's emotions, means for the emotion engine to extract emotions from the user's utterances and detect negative emotions, and means for suggesting to the crew members a response according to the emotions detected by the emotion engine. This enables the crew members to respond appropriately to the user's emotions, and is expected to improve customer satisfaction.

[0487] A "natural language processing model" is an algorithm or machine learning model that analyzes crew talk and determines what is good and what is bad.

[0488] An "emotion engine" is an algorithm or machine learning model that extracts emotions from user comments and detects negative emotions.

[0489] A "highly productive crew" is a crew that achieves high results in customer service.

[0490] A "low-productivity crew" refers to a crew member who achieves low results in customer service.

[0491] "Talk" refers to the phrases, expressions, and conversation flow that crew members use while interacting with customers.

[0492] "Correction" refers to the process of analyzing the crew's talk and pointing out areas for improvement.

[0493] "User" refers to a customer or user who interacts with the crew.

[0494] "Response suggestion" refers to the process of suggesting appropriate responses to the crew based on the emotions detected by the emotion engine.

[0495] This invention is a system that analyzes crew chatter, recognizes the user's emotions, and suggests appropriate responses. This system operates in cooperation with a server, a terminal, and a user.

[0496] System configuration

[0497] The server includes the following major components:

[0498] 1. Natural language processing models: These are algorithms or machine learning models that analyze crew talk and determine what is good and what is bad. Specifically, we use generative AI models such as BERT and GPT-3 (registered trademarks).

[0499] 2. Sentiment engine: An algorithm or machine learning model that extracts emotions from user comments and detects negative emotions.

[0500] 3. Database: A database for storing crew talk data and user emotion data.

[0501] The terminal is a device used by the crew that provides an interface for receiving feedback and suggestions from the server in real time.

[0502] Users are customers or passengers who interact with the crew, and their comments are analyzed by the system.

[0503] System Operation

[0504] 1. Data Collection:

[0505] The server collects voice data of phrases, expressions, and conversation flow used by crew members while interacting with customers.

[0506] The server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the voice data into text data.

[0507] 2. Data Preprocessing:

[0508] The server removes unnecessary noise from the collected text data and extracts the important parts.

[0509] The server uses natural language processing libraries (e.g., NLTK, spaCy) to tokenize and tag the text data.

[0510] 3. Model training:

[0511] The server trains a natural language processing model using the preprocessed data.

[0512] The server compares the talk of high-productivity crews with that of low-productivity crews and learns the criteria for judging good and bad.

[0513] 4. Emotion recognition:

[0514] The server uses an emotion engine to recognize the user's emotion.

[0515] The server extracts emotions from users' comments and detects negative emotions such as anger and dissatisfaction.

[0516] 5. Proposed solutions:

[0517] The server suggests responses to the crew based on the recognized emotions.

[0518] The server displays the proposals on the crew's terminals in real time.

[0519] Specific examples

[0520] When a user says, "My order is late, what's going on?", the following happens:

[0521] 1. Data Collection:

[0522] The server collects the user's speech as audio data and converts it into text data using the Google Speech-to-Text API.

[0523] 2. Data Preprocessing:

[0524] The server tokenizes the text data and extracts the important parts.

[0525] 3. Model training:

[0526] The server uses the preprocessed data to train a generative AI model.

[0527] 4. Emotion recognition:

[0528] The server detects user dissatisfaction using an emotion engine.

[0529] 5. Proposed solutions:

[0530] The server will suggest a response to the crew member's terminal, such as "We're sorry. We'll check right away."

[0531] Prompt Sentence Examples

[0532] "What action should we suggest to the crew when a user makes a statement that indicates dissatisfaction?"

[0533] In this way, the server learns the crew's talk data and provides a system that suggests responses in real time according to the user's emotions.

[0534] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0535] Step 1: Data collection

[0536] The server collects voice data of phrases, expressions, and conversation flow used by crew members while interacting with customers.

[0537] Input: Crew voice data

[0538] Data processing: Using voice recognition software (e.g., Google Speech-to-Text API), converting voice data into text data.

[0539] Output: Text data

[0540] What it does: When a crew member says, "Hello, how can I help you?", their voice is converted to text in real time.

[0541] Step 2: Data Preprocessing

[0542] The server removes unnecessary noise from the collected text data and extracts the important parts.

[0543] Input: Text data

[0544] Data processing: Use a natural language processing library (e.g., NLTK, spaCy) to tokenize and tag the text data with parts of speech.

[0545] Output: Preprocessed text data

[0546] What it does: The text "Hello, how can I help you?" is split into the tokens "Hello," "How can I help you?", "How can I help you?", and "How can I help you?".

[0547] Step 3: Model training

[0548] The server trains a natural language processing model using the preprocessed data.

[0549] Input: Preprocessed text data

[0550] Data calculation: Compare the talk of high-productivity crew members with that of low-productivity crew members and learn criteria for judging good and bad. The model used is a generative AI model such as BERT or GPT-3.

[0551] Output: Trained model

[0552] How it works: Using generative AI models like BERT and GPT-3, phrases like "Hello, how can I help you?" are learned as characteristics of a highly productive crew member.

[0553] Step 4: Emotion Recognition

[0554] The server uses an emotion engine to recognize the user's emotion.

[0555] Input: User's text data

[0556] Data calculation: The emotion engine extracts emotions from user comments and detects negative emotions such as anger or dissatisfaction.

[0557] Output: User emotion data

[0558] What it does: When a user says, "My order is late, what's going on?" the emotion engine detects dissatisfaction.

[0559] Step 5: Propose a response

[0560] The server suggests responses to the crew based on the recognized emotions.

[0561] Input: User emotion data

[0562] Data calculation: Generates appropriate responses based on the emotions detected by the emotion engine.

[0563] Output: Suggested action for crew

[0564] Specific action: A response such as "We're sorry. We'll check it right away" will be displayed in real time on the crew's terminal.

[0565] In this way, the server learns the crew's talk data and provides a system that suggests responses in real time according to the user's emotions.

[0566] (Application example 1)

[0567] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0568] When employees interact with customers, they are required to recognize their emotions appropriately and respond quickly and effectively. However, because this depends on the skills and experience of the employees, the quality of the response can vary. In addition, it is difficult to propose responses in real time, making it difficult to improve customer satisfaction.

[0569] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a natural language processing model for learning the dialogue between high-productivity employees and low-productivity employees, and includes means for the natural language processing model to analyze the employee dialogue and determine what is good and what is bad, means for the natural language processing model to correct the employee dialogue, means for recognizing customer emotions using the natural language processing model incorporating an emotion engine and suggesting to the employee a response based on the emotion, and means installed in the smart glasses to suggest a response to the employee in real time. This enables the employee to instantly recognize the customer's emotions and take an appropriate response in real time.

[0570] A "highly productive employee" is an employee who is highly efficient and effective in their interactions with customers.

[0571] "Low productivity employees" are employees who are inefficient and ineffective in their interactions with customers.

[0572] "Dialogue" is the verbal exchange that takes place between employees and customers.

[0573] A "natural language processing model" is an artificial intelligence technology for understanding and analyzing human language.

[0574] "Analysis" is the process of examining and evaluating the content of a dialogue in detail.

[0575] "Judgment" is the act of evaluating the quality of a dialogue and drawing a conclusion.

[0576] "Correction" is the act of correcting and improving the content of a dialogue.

[0577] An "emotion engine" is a technology for recognizing and analyzing customer emotions from conversations.

[0578] "Suggestion" is the act of suggesting how to respond to an employee based on perceived emotions.

[0579] "Smart glasses" are wearable devices that have the ability to display information and capture audio and video.

[0580] "Real-time" refers to immediate processing or response without delay.

[0581] This invention is a system that recognizes customer emotions in real time when employees interact with customers in a physical store and suggests appropriate responses. This system combines a natural language processing model and an emotion engine and is implemented using smart glasses.

[0582] System configuration

[0583] The system consists of the following main components:

[0584] 1. Natural language processing model: Learns the interactions between high- and low-productivity employees and determines whether the interaction is good or bad.

[0585] 2. Emotion engine: Recognizes emotions from customer interactions and suggests responses based on those emotions.

[0586] 3. Smart glasses: worn by employees, they display conversation content and suggested responses in real time.

[0587] Program processing

[0588] The server implements a natural language processing model and emotion engine using Python, spaCy, and the Transformers library, while the smart glasses are powered by wearable devices such as Google Glass.

[0589] 1. Data capture: The microphone in the smart glasses captures interactions between employees and customers.

[0590] 2. Data analysis: The captured dialogue data is sent to a server and analyzed using a natural language processing model.

[0591] 3. Emotion Recognition: The emotion engine recognizes customer emotions from conversation data.

[0592] 4. Response Suggestion: Suggest appropriate responses to employees based on perceived emotions.

[0593] 5. Real-time display: The suggested responses are displayed in real time on the smart glasses display.

[0594] Specific examples

[0595] For example, if a customer says, "I'm unhappy with the service I received recently," the emotion engine recognizes "unhappy" and suggests to an employee, "Sorry, sir. What is the problem?"

[0596] Prompt Sentence Examples

[0597] Customer: I'm unhappy with your recent service.

[0598] Employee: Sorry, sir. What is the problem?

[0599] This system allows employees to instantly recognize customer emotions and respond appropriately in real time, which is expected to improve customer satisfaction.

[0600] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0601] Step 1:

[0602] The microphone in the smart glasses captures the conversation between the employee and the customer. The input is the voice data of the employee and the customer, and the output is the captured voice data. The smart glasses transmit the voice data to the server in real time.

[0603] Step 2:

[0604] The server converts the received voice data into text data. The input is the captured voice data, and the output is the converted text data. It converts the voice data into text using speech recognition software (e.g., Google Speech-to-Text API).

[0605] Step 3:

[0606] The server analyzes the text data using a natural language processing model. The input is the converted text data, and the output is the analysis result. A natural language processing model (e.g., spaCy) is used to extract specific patterns and phrases from the text data and determine whether the dialogue is good or bad.

[0607] Step 4:

[0608] The server uses an emotion engine to recognize customer emotions from the analysis results. The input is the analysis results, and the output is the recognized emotion data. The emotion engine (e.g., emotion recognition model from the Transformers library) is used to extract customer emotions from text data.

[0609] Step 5:

[0610] The server proposes an appropriate response based on the recognized emotion data. The input is the recognized emotion data, and the output is the response proposal data. The server generates a specific response method for the employee based on a predefined response pattern.

[0611] Step 6:

[0612] The smart glasses display the response suggestion data received from the server to the employee in real time. The input is the response suggestion data, and the output is the response suggestion displayed on the smart glasses' display. The smart glasses visually display the response suggestion so that the employee can immediately confirm it.

[0613] Step 7:

[0614] The employee responds to the customer based on the response suggestions displayed on the smart glasses. The input is the response suggestions displayed on the smart glasses, and the output is the employee's specific response behavior. The employee then carries out the suggested response method and responds in accordance with the customer's feelings.

[0615] Example 2

[0616] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0617] Conventional crew conversation analysis systems have difficulty analyzing crew conversation content in real time and providing appropriate feedback. Furthermore, they lack the functionality to grasp users' emotions in real time and instruct crews on how to respond accordingly, making it difficult to improve customer satisfaction. This has prevented crew skills and the quality of customer service from being fully improved.

[0618] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0619] In this invention, the server includes means for receiving crew chat content in real time, means for transmitting the chat content to a natural language processing model to detect inappropriate expressions and provide correction instructions, and means for transmitting the chat content to a sentiment analysis engine to analyze the user's emotions and provide response instructions. This makes it possible to analyze the crew chat content in real time and provide appropriate feedback. Furthermore, by understanding the user's emotions in real time and instructing the crew to respond accordingly, customer satisfaction can be improved.

[0620] "Crew" refers to the person in charge of interacting with customers.

[0621] "Natural language processing model" refers to artificial intelligence technology that analyzes the content of crew conversations and provides appropriate feedback.

[0622] "Talk content" refers to the words and phrases used by crew members in their interactions with customers.

[0623] "Real-time" means that processing occurs at the exact moment the crew is interacting with the customer.

[0624] "Inappropriate language" refers to words or phrases that may be offensive or misleading to customers.

[0625] "Correction instructions" refer to instructions for changing inappropriate expressions to appropriate expressions.

[0626] An "emotion analysis engine" refers to technology that analyzes user emotions from chat content and provides the results.

[0627] "User" refers to a customer who interacts with a crew member.

[0628] "Response instructions" refers to instructing the crew to respond appropriately according to the user's emotions.

[0629] "Feedback" refers to providing an evaluation of the crew's talk content and suggestions for improvement.

[0630] MODE FOR CARRYING OUT THE INVENTION

[0631] The present invention is a system that analyzes the content of conversations between crew members and customers in real time and provides appropriate feedback. Specific embodiments of this system will be described below.

[0632] Hardware and software used

[0633] The server plays a central role in receiving and processing the crew's conversation content in real time. The server is a computer system equipped with a high-performance processor and sufficient memory. The terminal is the device through which the crew communicates with customers, and is typically a PC or tablet. The user is the customer who communicates with the crew.

[0634] The software uses a common generative AI model for natural language processing, specifically OpenAI's GPT-4, and IBM Watson's Tone Analyzer for sentiment analysis.

[0635] Program processing

[0636] The server receives the chat content entered by the crew members using their terminals in real time. The received chat content is first sent to a natural language processing model. The natural language processing model analyzes the chat content and determines whether it contains inappropriate language. If inappropriate language is detected, the server provides instructions to the crew members to correct it to appropriate language.

[0637] At the same time, the server also sends the chat content to a sentiment analysis engine, which extracts patterns and phrases from the chat that reflect the user's emotions. If the user uses expressions that indicate joy or satisfaction, the sentiment analysis engine detects this and the server instructs the user to take action to maintain positive emotions, such as expressing gratitude to the crew.

[0638] Specific examples

[0639] For example, if a crew member says to a customer, "This product might not look good on you," the natural language processing model can provide a corrective instruction such as, "I think this product would look great on you."

[0640] Or, if a user says, "This service is absolutely amazing!", the sentiment analysis engine will detect this and instruct the server to reply, "Thank you! Your satisfaction is our greatest joy."

[0641] Prompt Sentence Examples

[0642] "If a crew member uses inappropriate language with a customer, provide instructions to correct it to appropriate language."

[0643] "If the user uses expressions that indicate joy or satisfaction, instruct the crew member to express gratitude."

[0644] This system will enable crew members to communicate with customers more appropriately and effectively, which is expected to improve customer satisfaction.

[0645] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0646] Step 1:

[0647] The server receives the talk content entered by the crew using the terminal in real time.

[0648] Input: What the crew member types into the terminal (e.g., "This product may not suit you").

[0649] Output: The received chat content is stored as internal data.

[0650] Specific operation: The crew member inputs the content of the conversation into the terminal, and the terminal sends the data to the server. The server receives this data and proceeds to the next processing step.

[0651] Step 2:

[0652] The server sends the received talk content to a natural language processing model.

[0653] Input: The message received by the server

[0654] Output: The conversation sent to the natural language processing model

[0655] Specific operation: The server sends the received chat content to a natural language processing model, which analyzes the chat content and determines whether it contains inappropriate language.

[0656] Step 3:

[0657] A natural language processing model detects inappropriate language and generates correction instructions.

[0658] Input: The conversation sent to the natural language processing model

[0659] Output: Correction instructions (e.g., "I think this product looks great on you.")

[0660] Specific operation: The natural language processing model analyzes the content of the conversation and determines that the expression "It might not suit you" is inappropriate. The model then generates instructions to correct it to "I think it suits you very much" and returns them to the server.

[0661] Step 4:

[0662] The server sends correction instructions from the natural language processing model to the crew's terminal.

[0663] Input: Correction instructions from a natural language processing model

[0664] Output: Corrective instructions displayed on the crew's terminal

[0665] Specific operation: The server sends the correction instructions received from the natural language processing model to the crew's device, where the correction suggestions are displayed, and the crew member corrects the talk based on them.

[0666] Step 5:

[0667] The server sends the received chat content to a sentiment analysis engine.

[0668] Input: The message received by the server

[0669] Output: The conversation content sent to the sentiment analysis engine

[0670] Specific operation: The server sends the same chat content to the sentiment analysis engine and requests sentiment analysis.

[0671] Step 6:

[0672] The sentiment analysis engine analyzes the user's sentiment and returns the results to the server.

[0673] Input: The conversation content sent to the sentiment analysis engine

[0674] Output: User sentiment analysis results (e.g., positive sentiment)

[0675] How it works: The sentiment analysis engine analyzes the content of conversations and extracts patterns and phrases that reflect the user's emotions. If the user says "great," it detects positive emotions and returns the results to the server.

[0676] Step 7:

[0677] The server provides response instructions to the crew based on the sentiment analysis results.

[0678] Input: Sentiment analysis results from the sentiment analysis engine

[0679] Output: Instructions to be displayed on the crew's device (e.g. "Thank you! Your satisfaction is our greatest joy.")

[0680] Specific operation: Based on the emotion analysis results, the server generates response instructions for the crew and sends them to the crew's terminal. The response instructions are displayed on the crew's terminal, and the crew responds to the user based on the instructions.

[0681] In this way, crew members can interact with customers more appropriately and effectively.

[0682] (Application example 2)

[0683] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0684] With conventional crew conversation analysis systems, it was difficult to analyze crew conversation content in real time and provide appropriate advice. It was also difficult to grasp changes in customer emotions in real time and suggest appropriate responses to the crew. As a result, improvements in crew skills and customer satisfaction were not fully achieved.

[0685] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0686] In this invention, the server includes: means for including a natural language processing model for learning the talk of high-productivity crew members and low-productivity crew members; means for the natural language processing model to analyze the crew members' talk and determine what is good and what is bad; means for the natural language processing model to correct the crew members' talk; means for the natural language processing model to analyze the crew members' talk in real time and provide corrections and advice as needed; means for an emotion engine to extract patterns and phrases that reflect the user's emotions from the crew members' talk; means for the emotion engine to grasp changes in the user's emotions in real time and suggest corresponding responses to the crew members; and means for including an application to be installed on the smart glasses. This makes it possible to analyze the content of the crew members' talk in real time and provide appropriate advice, and further to grasp changes in customer emotions in real time and suggest corresponding responses to the crew members.

[0687] A "highly productive crew" is a staff member who is able to achieve high efficiency and results in customer service.

[0688] A "low-productivity crew" is a staff member who is inefficient and unproductive in dealing with customers.

[0689] A "natural language processing model" is an algorithm or machine learning model that analyzes text data and understands its meaning and context.

[0690] "Talk" refers to the words and expressions used by crew members when interacting with customers.

[0691] "Real-time analysis" refers to the process of instantly analyzing the content of crew conversations and providing the results immediately.

[0692] "Correction" refers to correcting the content of the crew's talk and changing it to appropriate expression.

[0693] "Advice" means instructions or guidance given to the crew.

[0694] An "emotion engine" is an algorithm or machine learning model that analyzes user emotions from text data and detects changes in those emotions.

[0695] A "pattern or phrase" is a combination of words or a sentence used to express a particular meaning or emotion.

[0696] "Smart glasses" are glasses-type devices that have built-in displays and sensors and have the ability to display information and collect data.

[0697] A system for implementing this invention is configured as follows: The server includes a natural language processing model for learning the talk of high-productivity and low-productivity crew members. The natural language processing model analyzes the crew members' talk and determines what is good and what is bad. The natural language processing model also analyzes the crew members' talk in real time and provides corrections and advice as needed. Furthermore, an emotion engine extracts patterns and phrases from the crew members' talk that reflect the user's emotions, grasps changes in the user's emotions in real time, and suggests appropriate responses to the crew members. These functions are provided through an application installed on the smart glasses.

[0698] The server uses a Python program to implement a natural language processing model and emotion engine. The natural language processing model uses the spacy library to analyze Japanese text. The emotion engine uses the emotion analysis model from the transformers library. The content of the crew's conversations is sent to the server in real time via smart glasses and is analyzed immediately.

[0699] Specifically, when a crew member talks with a customer, they wear smart glasses and input their conversation content. The server analyzes the input conversation content using a natural language processing model, detects and corrects inappropriate expressions, and uses an emotion engine to analyze the user's emotions and suggests responses to the crew member based on changes in emotions. This allows the crew member to respond appropriately in real time.

[0700] For example, if a crew member says, "Customer, this is inappropriate expression 1. What do you think?", the server analyzes the conversation and corrects "inappropriate expression 1" to "appropriate expression." If the server determines that the user's emotion is neutral, it provides the crew member with the advice to "continue responding normally."

[0701] An example of a prompt is as follows:

[0702] "Customer, this is an inappropriate expression, what do you think?"

[0703] In this way, crew members can respond appropriately in real time, which is expected to improve customer satisfaction.

[0704] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0705] Step 1:

[0706] The crew puts on the smart glasses and begins interacting with the customer.

[0707] Input: Crew talk

[0708] Output: Talk content input into smart glasses

[0709] Specific operation: The crew inputs the content of the talk through the microphone on the smart glasses.

[0710] Step 2:

[0711] The smart glasses transmit the inputted talk content to the server.

[0712] Input: Talk content entered into the smart glasses

[0713] Output: The chat content sent to the server

[0714] Specific operation: The smart glasses send the conversation content to the server via Wi-Fi or Bluetooth.

[0715] Step 3:

[0716] The server receives the content of the conversation and analyzes it using a natural language processing model.

[0717] Input: The chat content sent to the server

[0718] Output: Parsed chat content

[0719] Specific operation: The server uses the spacy library to analyze the chat content and detect inappropriate expressions.

[0720] Step 4:

[0721] The server corrects inappropriate expressions and converts them into appropriate expressions.

[0722] Input: Parsed chat content

[0723] Output: Modified chat content

[0724] Specific operation: The server replaces inappropriate expressions with appropriate expressions. For example, it corrects "inappropriate expression 1" to "appropriate expression."

[0725] Step 5:

[0726] The server analyzes the user's emotions using an emotion engine.

[0727] Input: Edited chat content

[0728] Output: User sentiment analysis results

[0729] What it does: The server uses a sentiment analysis model from the transformers library to classify the user's sentiment as positive, negative, or neutral.

[0730] Step 6:

[0731] The server suggests responses to the crew based on the user's emotions.

[0732] Input: User sentiment analysis results

[0733] Output: Advice for crew

[0734] Specific operation: If the user's emotion is positive, the server generates advice such as "Express gratitude," if negative, "Respond politely," and if neutral, "Continue responding normally."

[0735] Step 7:

[0736] The server sends the modified talk content and advice to the smart glasses.

[0737] Input: Revised talk, advice to crew

[0738] Output: Corrective speech and advice sent to smart glasses

[0739] Specific operation: The server sends the corrected talk content and advice to the smart glasses via Wi-Fi or Bluetooth.

[0740] Step 8:

[0741] Smart glasses display modified talk and advice to the crew.

[0742] Input: Corrected speech and advice sent to smart glasses

[0743] Output: Corrective talk and advice displayed to the crew

[0744] Specific operation: The revised talk content and advice are displayed on the smart glasses display, and the crew confirms it.

[0745] In this way, crews can take appropriate action in real time.

[0746] Example 3

[0747] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0748] Conventional systems for improving crew talking skills do not adequately analyze the content of crew conversations or recognize the user's emotions, making it difficult for crew members to objectively understand their own talking skills and take concrete measures to improve them. Furthermore, there was a lack of feedback on the user's emotions, making it difficult for crew members to understand how they influenced the user's emotions. This led to the issue of ineffective improvement of crew talking skills.

[0749] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0750] In this invention, the server includes means for collecting crew talk data and transmitting it to the server, means for analyzing the user's emotions using an emotion engine for recognizing the user's emotions, and means for generating feedback by integrating areas for strengthening or improving talk skills and the user's emotion recognition results. This allows crew members to objectively understand their own talk skills and take specific measures to improve them. Furthermore, by receiving feedback on the user's emotions, crew members can understand how they affected the user's emotions and use the results to improve their skills.

[0751] A "highly productive crew" is a crew that can achieve high results in their work.

[0752] A "low-productivity crew" refers to a crew member who performs poorly in their work.

[0753] A "natural language processing model" refers to an algorithm or system for analyzing and understanding natural language.

[0754] "Talk data" refers to voice data and text data generated when a crew member converses with a user.

[0755] "Server" refers to a computer system for processing and storing data.

[0756] An "emotion engine" refers to an algorithm or system for analyzing and recognizing a user's emotions.

[0757] "Feedback" refers to information that integrates areas for strengthening and improving the crew's talking skills and the results of user emotion recognition.

[0758] "Talk skills" refers to the communication ability of a crew member when interacting with a user.

[0759] "Areas of improvement" refers to the areas in which the crew's talking skills are particularly strong.

[0760] "Areas for improvement" refers to areas in the crew's speaking skills that need improvement.

[0761] "User emotion" refers to the emotional state that a user exhibits during an interaction.

[0762] "Analysis" refers to the process of examining data in detail and extracting meaning and patterns.

[0763] "Integration" refers to the process of bringing together multiple pieces of information.

[0764] This invention is a system for improving crew members' conversation skills, which uses a natural language processing model and an emotion engine to analyze the content of crew members' conversations and provide feedback. A specific embodiment of this system will be described below.

[0765] Hardware and software used

[0766] Hardware: Servers, terminals

[0767] Software: Natural language processing models (e.g., GPT-4), emotion engines (e.g., IBM Watson Tone Analyzer)

[0768] System configuration

[0769] 1. Device: When the crew interacts with the user, the voice and text data are collected. For example, when the crew says, "This product is very high performance," the voice is captured by the device's microphone.

[0770] 2. Terminal: The collected data is converted into text format and sent to the server. In the case of voice data, it is converted into text using voice recognition software (e.g., Google Speech-to-Text).

[0771] 3. Server: Inputs the received text data into a natural language processing model (e.g., GPT-4). For example, it analyzes the text "This product is very high performance."

[0772] 4. Server: A natural language processing model analyzes the conversation and extracts areas for skill improvement and strengthening. For example, it generates feedback such as, "You need to provide more details about the specific features of your product."

[0773] 5. Server: Use an emotion engine (e.g. IBM Watson Tone Analyzer) to recognize the user's emotion. For example, if the user says "But it's expensive," input that text into the emotion engine.

[0774] 6. Server: The emotion engine analyzes the user's emotions and recognizes emotions such as dissatisfaction and joy. For example, it gives a result such as "The user is dissatisfied with the price."

[0775] 7. Server: Integrates the points for strengthening and improving the talking skill and the results of user emotion recognition to generate feedback. For example, it creates specific advice such as, "To resolve the user's dissatisfaction, it would be a good idea to also explain the cost-effectiveness of the product."

[0776] 8. Server: Sends the generated feedback to the device.

[0777] 9. Terminal: The crew receives feedback and uses it to improve their speaking skills. For example, the terminal screen might say, "To resolve the user's complaints, it would be a good idea to also explain the cost-effectiveness of the product."

[0778] Examples and prompts

[0779] Examples:

[0780] Crew talk: "Customer, this product is very high performance and will last a long time."

[0781] User emotion recognition result: Dissatisfaction (e.g., the user is dissatisfied with the price)

[0782] Feedback: "To alleviate customer dissatisfaction, it would be a good idea to explain the cost-effectiveness of the product."

[0783] Example prompt sentence:

[0784] "Analyze the crew's conversations and identify areas for skill strengthening and improvement."

[0785] "Recognize user emotions and analyze how crew talk influences them."

[0786] In this way, the system supports the crew in improving their talking skills. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0787] Step 1:

[0788] Collecting crew talk data

[0789] Device: When the crew interacts with the user, the voice and text data are collected in real time. For example, when the crew says, "This product is very high performance," the voice is captured by the device's microphone.

[0790] Input: Crew voice data

[0791] Output: Text conversion result of audio data

[0792] What it does: Your device's microphone captures your voice and converts it into text using speech recognition software (e.g., Google Speech-to-Text).

[0793] Step 2:

[0794] Sending chat data

[0795] Terminal: Sends collected text data to the server.

[0796] Input: Text data converted by speech recognition software

[0797] Output: Text data sent to the server

[0798] Specific operation: The terminal sends text data to the server via the network.

[0799] Step 3:

[0800] Talk data analysis

[0801] Server: Inputs the received text data into a natural language processing model (e.g., GPT-4). For example, it analyzes the text "This product is very high performance."

[0802] Input: Text data sent to the server

[0803] Output: Analysis result of the talk content

[0804] What it does: Natural language processing models analyze text data and identify areas for skill enhancement and improvement.

[0805] Step 4:

[0806] emotion recognition

[0807] Server: Use an emotion engine (e.g. IBM Watson Tone Analyzer) to recognize the user's emotion. For example, if the user says "But it's expensive," input that text into the emotion engine.

[0808] Input: User's text data

[0809] Output: User emotion recognition result

[0810] Specific operation: The emotion engine analyzes text data and recognizes the user's emotions.

[0811] Step 5:

[0812] Generate feedback

[0813] Server: Generates feedback by integrating the results of the conversation skill's strengthening and improvement and the user's emotion recognition. For example, it creates specific advice such as, "To resolve the user's dissatisfaction, it would be a good idea to also explain the cost-effectiveness of the product."

[0814] Input: Analysis results of chat content, user emotion recognition results

[0815] Output: The generated feedback

[0816] Specific operation: The server integrates the analysis results and emotion recognition results and generates feedback.

[0817] Step 6:

[0818] Providing feedback

[0819] Server: Sends the generated feedback to the device.

[0820] Input: Generated feedback

[0821] Output: Feedback sent to the device

[0822] Specific operation: The server sends feedback to the device via the network.

[0823] Step 7:

[0824] Receiving and viewing feedback

[0825] Terminal: Crew members receive feedback to help them improve their speaking skills. For example, the terminal screen displays a message saying, "To resolve user complaints, it would be a good idea to also explain the cost-effectiveness of the product."

[0826] Input: Feedback sent by the server

[0827] Output: Feedback displayed to the crew

[0828] What happens: The device receives the feedback and displays it on the screen.

[0829] (Application example 3)

[0830] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0831] Conventional systems for improving crew conversation skills are limited to analyzing and correcting the content of crew conversations, and lack the ability to recognize and provide feedback on customer emotions in real time. This makes it difficult for crew members to immediately understand how their conversations have affected customers, resulting in a decrease in the efficiency of skill improvement.

[0832] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0833] In this invention, the server includes a natural language processing model for learning crew talk, means for the natural language processing model to analyze the crew talk and determine what is good and what is bad, means for the natural language processing model to correct the crew talk, means for feeding back the correction results of the crew talk to the crew and promoting the improvement of the crew's skills, an emotion recognition engine for recognizing customer emotions, means for the emotion recognition engine to analyze customer emotions and feed back the results to the crew, and means for displaying the feedback in real time on a smart device worn by the crew, thereby enabling the crew to instantly understand how their talk affected the customer and efficiently improve their skills.

[0834] A "natural language processing model" is a machine learning algorithm that analyzes crew talk and determines what is good and what is bad.

[0835] "Crew" refers to employees who deal with customers in physical stores, etc.

[0836] "Talk" refers to what the crew says when interacting with customers.

[0837] "Correction" is the act of correcting the crew's talk and pointing out areas for improvement and strengthening.

[0838] "Feedback" refers to the act of communicating the results of the analysis of the talk and the customer's emotion recognition to the crew.

[0839] An "emotion recognition engine" is an algorithm that analyzes a customer's facial expressions and tone of voice to recognize their emotions.

[0840] "Smart devices" refer to electronic devices worn by crew members that can display information in real time, including smart glasses and head-mounted displays.

[0841] "Real-time" means that information is provided at the exact moment the crew is interacting with the customer.

[0842] A system for implementing this invention includes a natural language processing model and an emotion recognition engine for improving crew members' speaking skills, and provides real-time feedback via smart devices worn by the crew members.

[0843] Hardware and software used

[0844] Hardware: Smart glasses, camera

[0845] Software: OpenCV (camera image acquisition and processing), Transformers (natural language processing model), emotion recognition engine

[0846] System processing overview

[0847] 1. Camera initialization and image acquisition

[0848] The server initializes the camera and captures real-time video footage of the interaction between the crew and the customer.

[0849] 2. Customer Emotion Recognition

[0850] The server detects the customer's face from the video frame and analyzes the customer's emotions using an emotion recognition engine, which analyzes the customer's facial expressions and tone of voice to recognize their emotions.

[0851] 3. Crew talk analysis

[0852] The server inputs the crew's conversation content into a natural language processing model and obtains the correction results. The natural language processing model analyzes the crew's conversation and determines what is good and what is bad.

[0853] 4. Providing Feedback

[0854] The server then provides feedback to the crew on the results of the conversation analysis and customer emotion recognition, which is displayed in real time on smart glasses worn by the crew.

[0855] Specific examples

[0856] For example, if a crew member asks, "Sir, what product are you looking for today?", the natural language processing model analyzes this speech and determines whether it is an appropriate phrase. At the same time, if the customer replies with a smile, "Thank you," the emotion recognition engine recognizes the customer's emotion as "happiness." These results are displayed in real time on the crew member's smart glasses, allowing the crew member to instantly understand how their speech affected the customer.

[0857] Prompt Sentence Examples

[0858] "Customer, what product are you looking for today?"

[0859] "Let me know if you need any help."

[0860] In this way, crew members receive real-time feedback and can efficiently improve their customer service skills.

[0861] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0862] Step 1:

[0863] The server initializes the camera and captures the interaction between the crew and the customer in real time. The input is the camera image and the output is the video frame. Specifically, OpenCV is used to capture the video stream from the camera and process each frame.

[0864] Step 2:

[0865] The server detects the customer's face from the video frame and analyzes the customer's emotion using an emotion recognition engine. The input is the video frame and the output is the customer's emotion data. Specifically, the emotion recognition engine detects the face in the frame and recognizes the emotion by analyzing facial expressions and tone of voice.

[0866] Step 3:

[0867] The server inputs the crew's talk content into a natural language processing model and obtains the correction results. The input is the crew's talk text, and the output is the correction results. Specifically, it uses Transformers to analyze the talk text and determine what is good and what is bad.

[0868] Step 4:

[0869] The server then feeds back the results of the conversation analysis and customer emotion recognition to the crew. The input is the conversation correction results and customer emotion data, and the output is feedback information. Specifically, the server generates data to display these results in real time on the smart glasses and provides it to the crew.

[0870] Step 5:

[0871] Crew members receive real-time feedback through the smart glasses to improve their customer service skills. The input is feedback information, and the output is improvements to the crew members' behavior. Specifically, the feedback information is displayed on the smart glasses' display, and the crew members use it to improve their conversations and responses.

[0872] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0873] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0874] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0875] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0876] [Second embodiment]

[0877] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0878] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0879] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0880] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0881] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0882] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0883] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0884] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0885] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0886] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0887] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0888] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0889] "Example 1"

[0890] In one embodiment of the present invention, the natural language processing model learns the conversations of highly productive and less productive crew members. Specifically, data such as phrases, expressions, and conversation flow used by crew members during conversations with customers is collected and used as training data. This training data serves as a criterion for judging the quality of a crew member's conversation.

[0891] "Example 2"

[0892] Furthermore, the natural language processing model corrects the crew's conversations. Specifically, when the crew interacts with a customer, the content of the conversation is analyzed in real time and corrections and advice are provided as needed. For example, if the crew uses an inappropriate expression while speaking to a customer, the natural language processing model detects the expression and provides instructions to the crew to correct it to an appropriate expression.

[0893] "Example 3"

[0894] The natural language processing model also provides feedback to help crew members improve their skills. Specifically, based on the results of the corrections to the crew's conversations, it clarifies areas for improvement and strengthens the crew's conversation skills, and provides this feedback to the crew. This allows the crew to objectively understand their own conversation skills and take concrete steps to improve them.

[0895] The processing flow of each embodiment will be described below.

[0896] "Example 1"

[0897] Step 1: The natural language processing model learns the conversations of high-productivity and low-productivity crew members. Specifically, it captures data such as phrases, expressions, and conversation flow used by crew members during conversations with customers.

[0898] Step 2: The captured data is used as training data to train a natural language processing model. Through this training, the model learns the criteria for judging whether a crew member's talk is good or bad.

[0899] "Example 2"

[0900] Step 1: The natural language processing model analyzes crew conversations in real time. Specifically, it captures the conversations that crew members make with customers in real time.

[0901] Step 2: Analyze the captured conversation and provide corrections or advice as needed. For example, if a crew member uses inappropriate language when speaking to a customer, the natural language processing model will detect that expression.

[0902] Step 3: Provide the crew with instructions to correct the detected inappropriate expressions into appropriate expressions. This allows the crew to improve their own talk in real time. "Example 3"

[0903] Step 1: Based on the results of the crew's talk corrections, the natural language processing model identifies areas for strengthening and improving the crew's talking skills.

[0904] Step 2: Provide feedback to the crew on the identified areas for improvement. This allows the crew to objectively understand their own speaking skills and take concrete steps to improve.

[0905] Example 1

[0906] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0907] In conventional crew dialogue evaluation systems, the criteria for judging the quality of dialogue were vague, making it difficult to effectively promote crew skill improvement. Furthermore, the steps of voice data text conversion, data preprocessing, feature extraction, model learning, evaluation, and feedback were not integrated, making efficient system operation difficult.

[0908] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0909] In this invention, the server includes means for converting voice data into text data, means for preprocessing the text data, means for extracting features from the preprocessed data, means for training a model using the extracted features, means for evaluating the performance of the trained model, means for improving the model based on the evaluation results, means for analyzing the crew's dialogue and determining what is good and what is bad, and means for correcting the crew's dialogue. This makes it possible to accurately determine the quality of the crew's dialogue and provide effective feedback.

[0910] A "highly productive crew" is a crew that can achieve high results in interactions with customers.

[0911] A "low-productivity crew" refers to a crew member who performs poorly in interactions with customers.

[0912] "Natural language processing model" refers to an algorithm or machine learning model for understanding and processing natural language.

[0913] "Dialogue" refers to the verbal exchange between the crew and the customer.

[0914] "Audio Data" refers to audio files that record the conversation between the crew and the customer.

[0915] "Text data" refers to data obtained by converting voice data into character information.

[0916] "Preprocessing" refers to the process of removing unnecessary information from text data and converting it into a format that is easy to analyze.

[0917] "Feature extraction" refers to the process of extracting useful information and patterns from text data.

[0918] "Model learning" refers to the process of training a machine learning model using the extracted features.

[0919] "Evaluation" refers to the process of measuring the performance of a trained model and confirming its accuracy and effectiveness.

[0920] "Feedback" refers to information provided to the crew members to promote skill improvement through evaluation results of the model and corrections to the dialogue.

[0921] The present invention is a system including a natural language processing model for learning the dialogue between a high-productivity crew and a low-productivity crew. A specific embodiment of this system will be described below.

[0922] System Overview

[0923] The user generates a program that uses a natural language processing model to learn the conversations between high-productivity and low-productivity crew members. This program takes in data such as phrases, expressions, and the flow of conversations used by crew members when interacting with customers, and uses this data as learning data.

[0924] Hardware and software used

[0925] Server: Collects data, preprocesses it, extracts features, trains it, evaluates it, and provides feedback.

[0926] Speech recognition software: Converts voice data into text data (e.g., Google Speech-to-Text API).

[0927] Natural language processing libraries: Preprocess text data (e.g., NLTK, spaCy).

[0928] Machine learning libraries: for feature extraction and model training (e.g., scikit-learn, TensorFlow, PyTorch).

[0929] Data processing and calculation

[0930] 1. Data Collection:

[0931] The server records the phrases and expressions used by the crew members while interacting with customers, as well as the flow of the conversation, and uses voice recognition software to convert the voice data into text data.

[0932] 2. Data Preprocessing:

[0933] The server preprocesses the collected text data, specifically by removing noise, tokenizing, and removing stop words.

[0934] 3. Feature Extraction:

[0935] The server extracts features from the preprocessed data, such as frequent phrases, sentiment analysis, and dialogue flow.

[0936] 4. Model training:

[0937] The server uses the extracted features to train a natural language processing model. The model is trained using the dialogue data of the high-productivity crew and the low-productivity crew as labeled data.

[0938] 5. Rating and Feedback:

[0939] The server evaluates the performance of the trained model using metrics such as precision, recall, and F1 score, improves the model based on the evaluation results, and provides feedback to the crew.

[0940] Specific examples

[0941] Example dialogue from a high-productivity crew: "Customers, this product is very popular. This feature is especially popular."

[0942] Example dialogue from a low-productivity crew: "Well, this product is okay. Nothing special."

[0943] Prompt Sentence Examples

[0944] "Generate a program to train a natural language processing model using the dialogue data of high-productivity and low-productivity crews. Specifically, the process will include the steps of collection, preprocessing, feature extraction, model training, and evaluation."

[0945] In this way, users can build a system that uses a natural language processing model to learn criteria for judging whether a crew member's dialogue is good or bad.

[0946] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0947] Step 1: Data collection

[0948] Input: Voice data of conversation between crew and customer

[0949] What it does: The server records the phrases and expressions used by the crew members while interacting with the customer, as well as the flow of the conversation. It then converts the voice data into text using speech recognition software (e.g., Google Speech-to-Text API).

[0950] Output: Text data

[0951] Step 2: Data Preprocessing

[0952] Input: Collected text data

[0953] Specific operation: The server preprocesses the collected text data, specifically removing noise, tokenizing, and removing stop words. This process uses natural language processing libraries (e.g., NLTK, spaCy).

[0954] Output: Preprocessed text data

[0955] Step 3: Feature extraction

[0956] Input: Preprocessed text data

[0957] Specific operation: The server extracts features from the preprocessed data, such as frequently used phrases, sentiment analysis, and dialogue flow. This process uses machine learning libraries (e.g., scikit-learn, TensorFlow).

[0958] Output: Extracted feature data

[0959] Step 4: Model training

[0960] Input: Extracted feature data and label data (interaction data of high-productivity crew and low-productivity crew)

[0961] Specific operation: The server uses the extracted features to train a natural language processing model. The conversation data of the high-productivity crew and the low-productivity crew are used as labeled data to train the model. This process uses a deep learning framework (e.g., PyTorch, TensorFlow).

[0962] Output: Trained model

[0963] Step 5: Rating and feedback

[0964] Input: trained model and test data

[0965] Specific operation: The server evaluates the performance of the trained model using metrics such as precision, recall, and F1 score, improves the model based on the evaluation results, and provides feedback to the crew.

[0966] Output: Evaluation results and feedback information

[0967] In this way, users can build a system that uses a natural language processing model to learn criteria for judging whether a crew member's dialogue is good or bad.

[0968] (Application example 1)

[0969] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0970] In conventional customer service work, there was a problem in which variations in crew conversation skills affected customer satisfaction. In particular, the conversations of low-productivity crew were often ineffective in communicating with customers, which led to lower sales and customer satisfaction. Another issue was the lack of feedback and training for crew to improve their conversation skills. To solve these problems, a system was needed that would help crew use appropriate phrases and expressions in real time and promote the improvement of their conversation skills.

[0971] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0972] In this invention, the server includes a natural language processing model for learning the talk of high-productivity crew members and low-productivity crew members, means for the natural language processing model to analyze the crew members' talk and determine what is good and what is bad, means for the natural language processing model to correct the crew members' talk, means for suggesting appropriate phrases and expressions in real time when the crew members interact with customers, means for evaluating the crew members' talk and providing feedback, and means for the crew members to use during training to acquire talking skills, thereby enabling the crew members to communicate effectively in interactions with customers, improving customer satisfaction and increasing sales.

[0973] A "highly productive crew" is an employee who has excellent communication skills when interacting with customers and is highly efficient at work.

[0974] "Low-productivity crew" refers to employees who lack communication skills when interacting with customers and have low work efficiency.

[0975] A "natural language processing model" is a model that uses algorithms and techniques to understand and generate human language.

[0976] "Talk" refers to the words, phrases, and expressions used by crew members when interacting with customers.

[0977] "Analysis" refers to the process of capturing crew talk as data and evaluating its content.

[0978] "Judgment" refers to the process of evaluating the quality of a crew member's talk based on evaluation criteria.

[0979] "Correction" is the process of correcting the crew's talk and changing it to more appropriate expressions.

[0980] "Real-time" means responding instantly to the moment the crew is interacting with the customer.

[0981] "Phrases" are short words or expressions that crew members use when interacting with customers.

[0982] "Evaluation" is the process of measuring the effectiveness and appropriateness of the crew's talk based on its content.

[0983] "Feedback" is the process of communicating the results of the crew's talk evaluation and pointing out areas for improvement and good points.

[0984] "Training" refers to the learning and practice that crew members undertake to improve their skills.

[0985] "Skills" refer to the techniques and abilities that crew members use in interacting with customers.

[0986] As an embodiment of the present invention, the following system can be constructed.

[0987] System Overview

[0988] The system suggests appropriate phrases and expressions in real time when the crew interacts with customers, and provides evaluations and feedback on the conversation. The system consists of a server and a terminal including smart glasses.

[0989] Hardware and software used

[0990] Hardware:

[0991] Smart glasses: equipped with a display and microphone, worn by the crew.

[0992] Server: A cloud server to host the natural language processing model and process data in real time.

[0993] software:

[0994] spaCy: A natural language processing library.

[0995] Transformers: Hugging Face's Transformers model library.

[0996] GPT-2: A generative AI model.

[0997] Data processing and calculation

[0998] 1. Talk Analysis:

[0999] Voice data from crew members interacting with customers is collected by the microphone in the smart glasses and converted into text data.

[1000] The server analyzes the collected text data using spaCy and evaluates the content of the talk.

[1001] 2. Real-time support:

[1002] The crew's input text is fed into the GPT-2 model to generate appropriate phrases and expressions.

[1003] The generated phrases and expressions are displayed in real time on the smart glasses' display.

[1004] 3. Talk evaluation and feedback:

[1005] The server evaluates the crew's talk and calculates a score.

[1006] The evaluation results are provided as feedback to the crew and used in training mode.

[1007] Specific examples

[1008] When the crew member asks, "Hello, customer. What products are you looking for today?", the smart glasses' display will show, "We have a new collection in stock. Please take a look."

[1009] Prompt Sentence Examples

[1010] Customer: Hello, do you have any recommendations for me today?

[1011] Crew: Hello, customer. What product are you looking for today?

[1012] In this way, the system supports crew conversations in real time, enabling smoother dialogue with customers, and can also encourage crew skill improvement through conversation evaluation and feedback.

[1013] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1014] Step 1:

[1015] The crew member begins a conversation with the customer.

[1016] Input: Crew voice.

[1017] Specific operation: The crew puts on the smart glasses and starts talking to the customer. The microphone in the smart glasses collects the crew's voice.

[1018] Step 2:

[1019] The terminal converts the voice data into text data.

[1020] Input: Crew voice data.

[1021] Data processing: Converting voice data into text using speech recognition software.

[1022] Output: Crew text data.

[1023] Specific operation: The built-in microphone of the smart glasses collects voice data and converts it into text data in real time.

[1024] Step 3:

[1025] The server analyzes the text data and generates appropriate phrases and expressions.

[1026] Input: Crew text data.

[1027] Data Computation: Analyzes text data using a natural language processing model (spaCy) and generates appropriate phrases and expressions using a generative AI model (GPT-2).

[1028] Output: Suggested phrases or expressions.

[1029] How it works: The server receives the text data, analyzes it, and then uses a generative AI model to generate appropriate phrases and expressions.

[1030] Step 4:

[1031] The device displays the generated phrases and expressions to the crew in real time.

[1032] Input: A suggested phrase or expression.

[1033] Output: A phrase or expression that will be displayed on the smart glasses display.

[1034] What it does: The generated phrases and expressions are displayed in real time on the smart glasses display, allowing the crew member to refer to them as they continue their conversation with the customer.

[1035] Step 5:

[1036] The server rates the crew's talk and provides feedback.

[1037] Input: Crew text data.

[1038] Data computation: Using natural language processing models to evaluate the content of the talk and calculate a score.

[1039] Output: Evaluation results as feedback.

[1040] What it does: The server evaluates the crew's talk, calculates a score, and generates feedback.

[1041] Step 6:

[1042] Crew members receive feedback and improve their skills in training mode.

[1043] Input: Evaluation results as feedback.

[1044] Output: Crew skill improvement.

[1045] What it does: Crew members receive feedback and practice to improve their speaking skills in training mode.

[1046] Example 2

[1047] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1048] Conventional dialogue support systems have the problem that it is difficult for workers to receive appropriate feedback on the spot when they use inappropriate expressions during a conversation with a customer. Additionally, there is a lack of specific advice to improve the quality of the conversation, which means that workers are unable to sufficiently improve their skills.

[1049] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1050] In this invention, the server includes means for analyzing the worker's dialogue in real time and determining what is good and what is bad, means for correcting the worker's dialogue, means for detecting inappropriate expressions and providing the worker with instructions to correct them to appropriate expressions, means for transmitting the generated advice to the worker's terminal, and means for the terminal to display the advice to the worker. This enables the worker to receive appropriate feedback in real time during a dialogue with a customer, and by providing specific advice to improve the quality of the dialogue, the worker's skills can be improved.

[1051] A "highly productive worker" refers to a worker who achieves high efficiency and results in their work.

[1052] "Low productivity workers" refer to workers who are low in efficiency and produce low results in their work.

[1053] A "natural language processing model" refers to an algorithm or machine learning model that understands and generates human language.

[1054] "Dialogue" refers to communication between a worker and a customer.

[1055] "Real-time analysis" refers to analyzing data instantly while a conversation is taking place.

[1056] "Inappropriate language" refers to words or phrases that may be offensive or misleading to customers.

[1057] "Appropriate expressions" refer to words and phrases that leave a favorable impression on customers and convey accurate information.

[1058] "Advice" refers to specific instructions or suggestions for the worker to improve the interaction.

[1059] "Terminal" refers to a computer or mobile device used by a worker.

[1060] "Feedback" refers to evaluations and information on areas for improvement provided to workers.

[1061] "Skill improvement" refers to improving workers' work abilities and communication skills.

[1062] The present invention provides a system for providing appropriate feedback in real time when a worker is interacting with a customer. The system includes a server, a terminal, and a natural language processing model.

[1063] Hardware and software used

[1064] Hardware: Servers, terminals (devices used by workers)

[1065] Software: Natural language processing models (e.g., GPT-4), real-time data analysis tools (e.g., Apache Kafka)

[1066] System Operation

[1067] 1. The user (worker) initiates a conversation with the customer. The worker uses a device (e.g., a tablet or PC) to communicate with the customer. The conversation is recorded on the device via voice input or text input.

[1068] 2. The device sends the conversation content to the server in real time using the WebSocket or HTTP protocol.

[1069] 3. The server inputs the received dialogue into the natural language processing model. The input data is in text format, and is preprocessed as necessary.

[1070] 4. The natural language processing model analyzes the dialogue and identifies inappropriate expressions and areas for improvement, such as "This product may not be suitable for you."

[1071] 5. The server receives the analysis results from the natural language processing model and generates appropriate expressions and advice, such as "You might want to compare this product with other options."

[1072] 6. The server sends the generated advice to the device using the WebSocket or HTTP protocol.

[1073] 7. The terminal displays the received advice to the user (worker) in the form of a pop-up or notification.

[1074] 8. The user (worker) modifies the dialogue based on the displayed advice. For example, they might say, "You might want to compare this product with other options."

[1075] Specific examples

[1076] Scenario: A worker says to a customer, "This product may not be right for you."

[1077] Analysis result of natural language processing model: "Inappropriate expressions were detected. These may create a negative impression on customers."

[1078] Generated advice: "You might want to compare this product with other options."

[1079] Prompt Sentence Examples

[1080] Input prompt: "When a worker says to a customer, 'This product may not be right for you,' please provide advice on how to revise it to an appropriate phrase."

[1081] Sample output: "You might want to compare this product with other options."

[1082] The system provides real-time support to help workers smoothly advance their interactions with customers.

[1083] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1084] Step 1:

[1085] The user (worker) starts a dialogue with the customer.

[1086] Input: The worker uses the terminal to interact with the customer.

[1087] Specific operation: The worker records the conversation content on the terminal through voice input or text input.

[1088] Output: The conversation is logged to the terminal.

[1089] Step 2:

[1090] The terminal transmits the contents of the conversation to the server in real time.

[1091] Input: Dialogue recorded on device.

[1092] Specific operation: The device captures the conversation content in real time and sends it to the server using WebSocket or HTTP protocol.

[1093] Output: The dialogue is sent to the server.

[1094] Step 3:

[1095] The server inputs the received dialogue content into a natural language processing model.

[1096] Input: The conversation sent to the server.

[1097] Specific operation: The server inputs the received dialogue content in text format into the natural language processing model, and performs preprocessing as necessary.

[1098] Output: The dialogue input to the natural language processing model.

[1099] Step 4:

[1100] A natural language processing model analyzes the dialogue and detects inappropriate expressions and areas for improvement.

[1101] Input: The dialogue that is fed into the natural language processing model.

[1102] How it works: The natural language processing model analyzes the conversation and identifies inappropriate expressions and areas for improvement, such as "This product may not be suitable for you."

[1103] Output: Detected inappropriate expressions and areas for improvement.

[1104] Step 5:

[1105] The server receives the analysis results from the natural language processing model and generates appropriate expressions and advice.

[1106] Input: Analysis results from a natural language processing model.

[1107] Specific operation: The server generates appropriate expressions and advice based on the analysis results. For example, it generates advice such as "You may want to compare this product with other options."

[1108] Output: The generated advice.

[1109] Step 6:

[1110] The server sends the generated advice to the terminal.

[1111] Input: The generated advice.

[1112] Specific operation: The server sends the generated advice to the device using WebSocket or HTTP protocol.

[1113] Output: Advice is sent to the terminal.

[1114] Step 7:

[1115] The terminal displays advice to the crew.

[1116] Input: Advice sent to the terminal.

[1117] Specific operation: The terminal displays the received advice to the worker in the form of a pop-up or notification.

[1118] Output: The advice displayed to the worker.

[1119] Step 8:

[1120] The user (operator) corrects the dialogue based on the displayed advice.

[1121] Input: The advice shown.

[1122] Specific behavior: The worker corrects the dialogue based on the displayed advice, for example, saying, "You might want to compare this product with other options."

[1123] Output: The corrected dialogue.

[1124] (Application example 2)

[1125] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1126] In traditional customer service work in brick-and-mortar stores, the content of the conversation between workers has a significant impact on customer satisfaction, but if a worker uses inappropriate language, it is difficult to correct it on the spot. Also, in order to improve the worker's skills, the content of the conversation must be fed back after the fact, making it difficult to improve in real time. This has led to issues such as a decrease in customer satisfaction and delays in improving worker skills.

[1127] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1128] In this invention, the server includes means for analyzing the worker's dialogue and determining what is good and what is bad, means for correcting the worker's dialogue, means for analyzing the worker's dialogue in real time and providing instructions for correcting to appropriate expressions, and means for displaying the corrected expressions on a display device. This makes it possible to correct to appropriate expressions in real time even if the worker uses inappropriate expressions during a dialogue with a customer, thereby improving customer satisfaction and worker skills at the same time.

[1129] A "highly productive worker" is a worker who achieves high efficiency and results in their work.

[1130] A "low-productivity worker" is a worker who is inefficient or produces poor results in their work.

[1131] "Dialogue" refers to verbal exchanges between a worker and a customer.

[1132] A "natural language processing model" is a model that uses artificial intelligence technology to understand, analyze, and generate human language.

[1133] "Analysis" is the process of examining the content of a conversation in detail and determining what is good and what is bad.

[1134] "Correction" refers to correcting the content of a conversation and changing it to appropriate expression.

[1135] "Real-time" refers to processing and analysis occurring at the exact moment the interaction is taking place.

[1136] "Appropriate expression" means using correct language that is not rude or misleading to customers.

[1137] "Instructions" are specific advice or orders given to workers to correct their behavior or words.

[1138] "Display device" means a device for visually displaying modified expressions or instructions.

[1139] A system for implementing the present invention includes smart glasses worn by a worker, a microphone for recognizing voice, a server for executing a natural language processing model, and a display device. A specific embodiment of this system will be described below.

[1140] First, a worker wears smart glasses and interacts with a customer through a microphone. The microphone captures the worker's voice in real time and sends the voice data to a server. The server then converts the voice data into text data using speech recognition software (e.g., the SpeechRecognition library).

[1141] The server then inputs the converted text data into a natural language processing model (e.g., the BERT model from the Transformers library) to analyze the dialogue. The natural language processing model determines whether the dialogue is appropriate and, if it contains inappropriate language, generates instructions to correct it to an appropriate expression.

[1142] The generated correction instructions are displayed in real time on the smart glasses' display device. This allows the worker to check the appropriate expression during the conversation and make immediate corrections. For example, if the worker says, "This product is cheap," the natural language processing model will determine that the expression "cheap" is negative, and the smart glasses will display a correction instruction to read, "This product is reasonably priced."

[1143] This system allows workers to learn appropriate expressions in real time while dealing with customers, thereby improving customer satisfaction and worker skills at the same time.

[1144] An example of a prompt is as follows:

[1145] "If a crew member says to a customer, 'This product is cheap,' generate instructions to correct that."

[1146] In this way, the quality of customer service provided by staff in physical stores can be improved.

[1147] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1148] Step 1:

[1149] The worker wears smart glasses and communicates with the customer through a microphone.

[1150] Input: Worker's voice

[1151] Output: Audio data

[1152] Specific action: The worker initiates a conversation with the customer, and the microphone captures their voice.

[1153] Step 2:

[1154] The audio data captured by the microphone is sent to the server.

[1155] Input: Audio data

[1156] Output: Audio data sent to the server

[1157] Specific operation: The audio data captured by the microphone is sent to the server in real time.

[1158] Step 3:

[1159] The server uses voice recognition software to convert the voice data into text data.

[1160] Input: Audio data

[1161] Output: Text data

[1162] Specific operation: The server uses the SpeechRecognition library to analyze the audio data and generate corresponding text data.

[1163] Step 4:

[1164] The server inputs text data into a natural language processing model and analyzes the content of the conversation.

[1165] Input: Text data

[1166] Output: Analysis results

[1167] Specific operation: The server analyzes the text data using the BERT model from the Transformers library and evaluates the appropriateness of the dialogue content.

[1168] Step 5:

[1169] If the server finds that an inappropriate expression is included, it generates instructions to correct it to an appropriate expression.

[1170] Input: Analysis results

[1171] Output: Correction instructions

[1172] Specific operation: Based on the analysis results of the natural language processing model, the server detects inappropriate expressions and generates instructions to correct them to appropriate expressions.

[1173] Step 6:

[1174] The server transmits the generated correction instructions to the display device of the smart glasses.

[1175] Input: Correction instructions

[1176] Output: Correction instructions displayed on the smart glasses

[1177] Specific operation: The server sends the correction instructions to the display device of the smart glasses so that the worker can check them.

[1178] Step 7:

[1179] The worker checks the correction instructions displayed on the display device of the smart glasses and corrects the content of the dialogue.

[1180] Input: Correction instructions displayed on smart glasses

[1181] Output: Corrected dialogue

[1182] Specific operations: The worker checks the correction instructions displayed on the smart glasses' display and corrects the content of the conversation with the customer to use appropriate expressions.

[1183] Example 3

[1184] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1185] Conventional systems for improving crew conversation skills have had problems such as insufficient analysis of conversation content and feedback, making it difficult for crew members to identify specific areas for improvement. Furthermore, saving conversation content and managing analysis results are often done manually, making it difficult to improve skills efficiently.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1187] In this invention, the server includes a means for the crew to input talk content, a means for the server to receive and store the talk content, a means for the server to send the talk content to a natural language processing model, a means for the natural language processing model to analyze the talk content, a means for the server to receive and store the analysis results, a means for the server to generate feedback, and a means for the crew to receive feedback, which makes it possible to efficiently analyze the talk content of the crew and provide specific feedback.

[1188] A "highly productive crew" is a crew that can achieve high results in their work.

[1189] A "low-productivity crew" refers to a crew member who performs poorly in their work.

[1190] A "natural language processing model" refers to an algorithm or machine learning model that analyzes the content of crew conversations and extracts areas for improvement or strengthening.

[1191] "Talk content" refers to the content of conversations and comments made by crew members while on duty.

[1192] "Server" refers to a computer system that receives and stores crew chat content and manages the analysis results.

[1193] "Feedback" refers to specific advice and areas for improvement provided to the crew based on the results of analysis by the natural language processing model.

[1194] A "prompt sentence" refers to an instruction sentence used when requesting analysis from a natural language processing model.

[1195] "Analysis results" refers to information on areas for enhancement or improvement obtained after the natural language processing model analyzes the content of the talk.

[1196] "Database" refers to an information management system for storing talk content and analysis results.

[1197] "Input form" refers to the interface that crew members use to enter talk content.

[1198] This invention is a system that provides feedback to improve crew members' speaking skills. The system analyzes the content of crew members' conversations using a natural language processing model, and provides feedback by clarifying areas for improvement.

[1199] Hardware and software used

[1200] Hardware: Servers, devices (PCs and smartphones)

[1201] Software: Natural language processing models (e.g., GPT-4), database management systems (e.g., MySQL), web application frameworks (e.g., Django)

[1202] Specific operation of the system

[1203] The user enters the content of the chat.

[1204] The user (crew member) uses the terminal to input the content of the conversation. For example, they might type "Good morning. How can I help you today?" into an input form on a web browser.

[1205] The server receives and stores the chat content.

[1206] The server receives the chat content sent from the device. The server saves the received chat content in a database (e.g., MySQL). When saving, it also records the user ID and timestamp.

[1207] The server sends the conversation content to the natural language processing model

[1208] The server sends the saved chat content to a natural language processing model (e.g., GPT-4). When sending, it generates a prompt and passes it to the model along with the chat content. For example, it generates a prompt like, "Please analyze the following chat content and provide feedback on areas for improvement and strengthening. Chat content: 'Good morning. What can we do for you today?'"

[1209] Natural language processing model analyzes the content of the conversation

[1210] The natural language processing model analyzes the content of the conversation based on the prompt received. As a result of the analysis, it extracts areas for improvement and strengthening. For example, it generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[1211] The server receives and stores the analysis results.

[1212] The server receives the analysis results from the natural language processing model and stores them in a database. When storing them, the server associates the conversation content with the corresponding analysis results.

[1213] The server generates feedback

[1214] The server generates feedback based on the analysis results. The feedback is provided in a format that is easy for the user to understand, and includes specific advice such as "next time, speak in a softer tone" or "use specific friendly words."

[1215] User receives feedback

[1216] The user can then use the device to review the generated feedback, which is displayed in a web browser. Based on the feedback, the user can take specific actions to improve their speaking skills.

[1217] Examples of specific examples and prompts

[1218] Examples:

[1219] The crew enters "how to greet customers."

[1220] The natural language processing model generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[1221] The crew will take steps to improve, such as "next time, speak in a softer tone" or "use specific, friendly words."

[1222] Example prompt sentence:

[1223] Please analyze the following conversation and provide feedback on areas for improvement. Conversation: 'Good morning. How can I help you today?'

[1224] In this way, the system supports the crew in improving their talking skills. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1225] Step 1:

[1226] The user enters the content of the chat.

[1227] The user (crew member) uses the terminal to input the content of the conversation. For example, they enter "Good morning. How can I help you today?" into the input form. The entered content of the conversation is sent to the system.

[1228] Input: The chat content entered by the user

[1229] Output: The message is sent to the server.

[1230] Step 2:

[1231] The server receives and stores the chat content.

[1232] The server receives the chat content sent from the device and saves it in a database (e.g., MySQL). When saving, it also records the user ID and timestamp.

[1233] Input: Chat content sent from the device

[1234] Output: The chat content is saved in the database.

[1235] Step 3:

[1236] The server sends the conversation content to the natural language processing model

[1237] The server sends the saved chat content to a natural language processing model (e.g., GPT-4). When sending, it generates a prompt and passes it to the model along with the chat content. For example, it generates a prompt like, "Please analyze the following chat content and provide feedback on areas for improvement and strengthening. Chat content: 'Good morning. What can we do for you today?'"

[1238] Input: Chat content stored in the database

[1239] Output: The conversation along with the prompt is sent to the natural language processing model.

[1240] Step 4:

[1241] Natural language processing model analyzes the content of the conversation

[1242] The natural language processing model analyzes the content of the conversation based on the prompt received. As a result of the analysis, it extracts areas for improvement and strengthening. For example, it generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[1243] Input: Prompt and talk content

[1244] Output: Analysis results (enhancements and improvements)

[1245] Step 5:

[1246] The server receives and stores the analysis results.

[1247] The server receives the analysis results from the natural language processing model and stores them in a database. When storing them, the server associates the conversation content with the corresponding analysis results.

[1248] Input: Analysis results from a natural language processing model

[1249] Output: Analysis results are saved in a database

[1250] Step 6:

[1251] The server generates feedback

[1252] The server generates feedback based on the analysis results. The feedback is provided in a format that is easy for the user to understand, and includes specific advice such as "next time, speak in a softer tone" or "use specific friendly words."

[1253] Input: Analysis results stored in a database

[1254] Output: Feedback is generated

[1255] Step 7:

[1256] User receives feedback

[1257] The user can then use the device to review the generated feedback, which is displayed in a web browser. Based on the feedback, the user can take specific actions to improve their speaking skills.

[1258] Input: Feedback from the server

[1259] Output: User receives and confirms feedback

[1260] (Application example 3)

[1261] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1262] Conventional systems for improving crew conversation skills have difficulty analyzing crew conversation content in real time and providing immediate feedback. Furthermore, the feedback is often not specific, making it unclear how crew members should improve. This leads to delays in crew skill improvement.

[1263] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1264] In this invention, the server includes a natural language processing model for learning the conversations of high-productivity workers and low-productivity workers, means for the natural language processing model to analyze the workers' conversations and determine what is good and what is bad, means for the natural language processing model to correct the workers' conversations, means for the natural language processing model to analyze the workers' conversations in real time and provide feedback on areas for improvement and strengthening, and means including a display device for displaying the feedback. This allows workers to receive specific feedback in real time, enabling them to improve their skills quickly and effectively.

[1265] A "highly productive worker" is a worker who achieves high efficiency and results in their work.

[1266] A "low-productivity worker" is a worker who is inefficient or produces poor results in their work.

[1267] A "natural language processing model" is a type of artificial intelligence technology for understanding and analyzing human language.

[1268] "Worker conversation" refers to verbal communication that occurs during work.

[1269] "Analysis" is the act of examining data or information in detail to find meaning and patterns.

[1270] "Judgment" is the act of deciding whether something is good or bad or appropriate based on information.

[1271] "Correction" is the act of correcting and improving errors in writing or conversation.

[1272] "Real-time analysis" means processing data instantly and obtaining results immediately.

[1273] "Areas of improvement" are points that can be used to further improve current skills and performance.

[1274] "Areas for improvement" are points in your current skills or performance that need to be corrected or improved.

[1275] "Feedback" is the act of providing evaluation or comment on behavior or results.

[1276] A "display device" is hardware for visually displaying information.

[1277] A system for implementing this invention is configured as follows: The server includes a natural language processing model for learning conversations between high-productivity workers and low-productivity workers. The natural language processing model analyzes the workers' conversations and determines what is good and what is bad. The natural language processing model also corrects the workers' conversations, analyzes them in real time, and provides feedback on areas for improvement and strengthening. The feedback is provided to the workers via a display device.

[1278] Hardware and software used

[1279] Hardware: Smartphones (e.g., iPhone, Android devices), smart glasses (e.g., Google Glass)

[1280] Software: Natural language processing models (e.g., OpenAI GPT-4), speech recognition software (e.g., Google Speech-to-Text, IBM Watson Speech to Text), feedback systems (e.g., custom feedback engines)

[1281] Data processing and calculation

[1282] 1. Voice recognition: The microphone on the device (smartphone or smart glasses) is used to record the worker's speech, which is then converted into text using voice recognition software.

[1283] 2. Natural language processing: The server inputs the converted text into a natural language processing model to analyze the conversation.

[1284] 3. Feedback generation: Based on the analysis results, the server generates enhancements and improvements in a feedback system.

[1285] 4. Real-time feedback: Display feedback on the device display.

[1286] Specific examples

[1287] For example, consider a situation where a staff member in a physical store is explaining a new product to a customer. If the staff member says, "This product is very good. Please try it," the natural language processing model will analyze this conversation and display feedback such as, "It would be good to explain the product's features in more detail. For example, it would be effective to give a specific example such as, 'This product uses the latest technology and is 30% more efficient than previous products.'"

[1288] Prompt Sentence Examples

[1289] Analyze the conversation of a staff member explaining a new product to a customer and generate feedback in the following format:

[1290] Conversation: [Staff conversation]

[1291] Feedback: [Improvements and improvements]

[1292] example:

[1293] Conversation: "This product is great. You should definitely try it."

[1294] Feedback: "It would be helpful to provide a more detailed description of the product's features. For example, you could say, 'This product uses cutting-edge technology and is 30% more efficient than conventional products.'"

[1295] In this way, workers receive real-time, specific feedback, enabling them to improve their skills quickly and effectively.

[1296] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1297] Step 1:

[1298] Voice Recognition

[1299] The microphone on the device (smartphone or smart glasses) is used to record the worker's conversation. The recorded voice data is input into voice recognition software (e.g., Google Speech-to-Text, IBM Watson Speech to Text) and converted into text data.

[1300] Input: Audio data

[1301] Output: Text data

[1302] Step 2:

[1303] Natural Language Processing

[1304] The server inputs the text data obtained in step 1 into a natural language processing model (e.g., OpenAI GPT-4) to analyze the conversation. The analysis extracts the content, tone, grammar, and specific patterns and phrases of the conversation.

[1305] Input: Text data

[1306] Output: Analysis results (evaluation of conversation content, specific patterns and phrases)

[1307] Step 3:

[1308] Feedback Generation

[1309] The server generates reinforcement and improvement points in the feedback system based on the analysis results obtained in step 2. The generated feedback includes specific advice and improvement measures.

[1310] Input: Analysis results

[1311] Output: Feedback data (enhancements and improvements)

[1312] Step 4:

[1313] Real-time feedback display

[1314] The terminal displays the feedback data generated in step 3 on a display device (such as a smartphone screen or smart glasses display). The worker receives the feedback in real time and can immediately implement improvements.

[1315] Input: Feedback data

[1316] Output: The displayed feedback

[1317] Specific operation example

[1318] For example, when a store staff member is explaining a new product to a customer, the process is as follows:

[1319] 1. Step 1: When the staff member says, "This product is great. Please try it," the voice is recorded by the microphone in the smart glasses and converted into text by speech recognition software.

[1320] 2. Step 2: The server inputs the converted text, "This product is very good. Please try it.", into a natural language processing model and analyzes the conversation. The analysis results point out the lack of specificity and the use of general phrases.

[1321] 3. Step 3: Based on the analysis results, the server generates feedback such as, "It would be better to explain the product's features in more detail. For example, it would be effective to provide a specific example such as, 'This product uses the latest technology and is 30% more efficient than conventional products.'"

[1322] 4. Step 4: The generated feedback is displayed on the smart glasses display, allowing staff to receive specific advice in real time.

[1323] Furthermore, an emotion engine that estimates the user's emotion may be combined.

[1324] The identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1325] "Example 1"

[1326] One embodiment of the present invention provides a system that uses a natural language processing model incorporating an emotion engine. This system recognizes the user's emotions from the crew's conversation and suggests appropriate responses to the crew based on those emotions. For example, if the user uses an expression that indicates anger or frustration, the emotion engine detects this and instructs the crew to respond to the user's emotions, such as by apologizing or proposing a solution.

[1327] "Example 2"

[1328] The emotion engine also extracts patterns and phrases from the crew's conversations that reflect the user's emotions. This makes it possible to grasp changes in the user's emotions in real time and suggest appropriate responses to the crew. For example, if the user uses expressions that show joy or satisfaction, the emotion engine will detect this and instruct the crew on how to maintain the user's positive emotions, such as expressing gratitude to the crew.

[1329] "Example 3"

[1330] Furthermore, the emotion engine provides feedback to the crew member on the results of corrections to their chat and the user's emotional recognition. This allows the crew member to understand how their chat affected the user's emotions and improve their skills based on the results. For example, if the crew member's proposed solution resolves the user's dissatisfaction and the user's emotions change to joy, the crew member can confirm through this feedback that their response was appropriate.

[1331] The processing flow of each embodiment will be described below.

[1332] "Example 1"

[1333] Step 1: The crew member initiates a conversation with the user.

[1334] Step 2: The natural language processing model analyzes the crew's talk, and the emotion engine recognizes the user's emotions.

[1335] Step 3: The system suggests a response to the crew based on the user's emotions recognized by the emotion engine.

[1336] "Example 2"

[1337] Step 1: The crew member initiates a conversation with the user.

[1338] Step 2: The emotion engine extracts patterns and phrases from the crew's conversation that reflect the user's emotions.

[1339] Step 3: Based on the patterns and phrases extracted by the emotion engine, the system suggests responses to the crew.

[1340] "Example 3"

[1341] Step 1: The crew member initiates a conversation with the user.

[1342] Step 2: The natural language processing model analyzes the crew's talk, and the emotion engine recognizes the user's emotions.

[1343] Step 3: The system provides feedback to the crew member on the results of corrections to their talk and the recognition of the user's emotions.

[1344] Step 4: Crew members use the feedback to improve their skills.

[1345] Example 1

[1346] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1347] Conventional crew conversation analysis systems lack the ability to not only judge the quality of a crew conversation, but also recognize the user's emotions in real time and suggest appropriate responses. This can lead to the crew not being able to respond appropriately to the user's emotions, which can result in a decline in customer satisfaction.

[1348] The identification processing by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a natural language processing model for learning the talk of high-productivity crew members and low-productivity crew members, means for the natural language processing model to analyze the crew members' talk and determine what is good and what is bad, means for the natural language processing model to correct the crew members' talk, an emotion engine for recognizing the user's emotions, means for the emotion engine to extract emotions from the user's utterances and detect negative emotions, and means for suggesting to the crew members a response according to the emotions detected by the emotion engine. This enables the crew members to respond appropriately to the user's emotions, and is expected to improve customer satisfaction.

[1349] A "natural language processing model" is an algorithm or machine learning model that analyzes crew talk and determines what is good and what is bad.

[1350] An "emotion engine" is an algorithm or machine learning model that extracts emotions from user comments and detects negative emotions.

[1351] A "highly productive crew" is a crew that achieves high results in customer service.

[1352] A "low-productivity crew" refers to a crew member who achieves low results in customer service.

[1353] "Talk" refers to the phrases, expressions, and conversation flow that crew members use while interacting with customers.

[1354] "Correction" refers to the process of analyzing the crew's talk and pointing out areas for improvement.

[1355] "User" refers to a customer or user who interacts with the crew.

[1356] "Response suggestion" refers to the process of suggesting appropriate responses to the crew based on the emotions detected by the emotion engine.

[1357] This invention is a system that analyzes crew chatter, recognizes the user's emotions, and suggests appropriate responses. This system operates in cooperation with a server, a terminal, and a user.

[1358] System configuration

[1359] The server includes the following major components:

[1360] 1. Natural language processing models: These are algorithms or machine learning models that analyze crew talk and determine what is good and what is bad. Specifically, we use generative AI models such as BERT and GPT-3.

[1361] 2. Sentiment engine: An algorithm or machine learning model that extracts emotions from user comments and detects negative emotions.

[1362] 3. Database: A database for storing crew talk data and user emotion data.

[1363] The terminal is a device used by the crew that provides an interface for receiving feedback and suggestions from the server in real time.

[1364] Users are customers or passengers who interact with the crew, and their comments are analyzed by the system.

[1365] System Operation

[1366] 1. Data Collection:

[1367] The server collects voice data of phrases, expressions, and conversation flow used by crew members while interacting with customers.

[1368] The server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the voice data into text data.

[1369] 2. Data Preprocessing:

[1370] The server removes unnecessary noise from the collected text data and extracts the important parts.

[1371] The server uses natural language processing libraries (e.g., NLTK, spaCy) to tokenize and tag the text data.

[1372] 3. Model training:

[1373] The server trains a natural language processing model using the preprocessed data.

[1374] The server compares the talk of high-productivity crews with that of low-productivity crews and learns the criteria for judging good and bad.

[1375] 4. Emotion recognition:

[1376] The server uses an emotion engine to recognize the user's emotion.

[1377] The server extracts emotions from users' comments and detects negative emotions such as anger and dissatisfaction.

[1378] 5. Proposed solutions:

[1379] The server suggests responses to the crew based on the recognized emotions.

[1380] The server displays the proposals on the crew's terminals in real time.

[1381] Specific examples

[1382] When a user says, "My order is late, what's going on?", the following happens:

[1383] 1. Data Collection:

[1384] The server collects the user's speech as audio data and converts it into text data using the Google Speech-to-Text API.

[1385] 2. Data Preprocessing:

[1386] The server tokenizes the text data and extracts the important parts.

[1387] 3. Model training:

[1388] The server uses the preprocessed data to train a generative AI model.

[1389] 4. Emotion recognition:

[1390] The server detects user dissatisfaction using an emotion engine.

[1391] 5. Proposed solutions:

[1392] The server will suggest a response to the crew member's terminal, such as "We're sorry. We'll check right away."

[1393] Prompt Sentence Examples

[1394] "What action should we suggest to the crew when a user makes a statement that indicates dissatisfaction?"

[1395] In this way, the server learns the crew's talk data and provides a system that suggests responses in real time according to the user's emotions.

[1396] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1397] Step 1: Data collection

[1398] The server collects voice data of phrases, expressions, and conversation flow used by crew members while interacting with customers.

[1399] Input: Crew voice data

[1400] Data processing: Using voice recognition software (e.g., Google Speech-to-Text API), converting voice data into text data.

[1401] Output: Text data

[1402] What it does: When a crew member says, "Hello, how can I help you?", their voice is converted to text in real time.

[1403] Step 2: Data Preprocessing

[1404] The server removes unnecessary noise from the collected text data and extracts the important parts.

[1405] Input: Text data

[1406] Data processing: Use a natural language processing library (e.g., NLTK, spaCy) to tokenize and tag the text data with parts of speech.

[1407] Output: Preprocessed text data

[1408] What it does: The text "Hello, how can I help you?" is split into the tokens "Hello," "How can I help you?", "How can I help you?", and "How can I help you?".

[1409] Step 3: Model training

[1410] The server trains a natural language processing model using the preprocessed data.

[1411] Input: Preprocessed text data

[1412] Data calculation: Compare the talk of high-productivity crew members with that of low-productivity crew members and learn criteria for judging good and bad. The model used is a generative AI model such as BERT or GPT-3.

[1413] Output: Trained model

[1414] How it works: Using generative AI models like BERT and GPT-3, phrases like "Hello, how can I help you?" are learned as characteristics of a highly productive crew member.

[1415] Step 4: Emotion Recognition

[1416] The server uses an emotion engine to recognize the user's emotion.

[1417] Input: User's text data

[1418] Data calculation: The emotion engine extracts emotions from user comments and detects negative emotions such as anger or dissatisfaction.

[1419] Output: User emotion data

[1420] What it does: When a user says, "My order is late, what's going on?" the emotion engine detects dissatisfaction.

[1421] Step 5: Propose a response

[1422] The server suggests responses to the crew based on the recognized emotions.

[1423] Input: User emotion data

[1424] Data calculation: Generates appropriate responses based on the emotions detected by the emotion engine.

[1425] Output: Suggested action for crew

[1426] Specific action: A response such as "We're sorry. We'll check it right away" will be displayed in real time on the crew's terminal.

[1427] In this way, the server learns the crew's talk data and provides a system that suggests responses in real time according to the user's emotions.

[1428] (Application example 1)

[1429] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1430] When employees interact with customers, they are required to recognize their emotions appropriately and respond quickly and effectively. However, because this depends on the skills and experience of the employees, the quality of the response can vary. In addition, it is difficult to propose responses in real time, making it difficult to improve customer satisfaction.

[1431] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a natural language processing model for learning the dialogue between high-productivity employees and low-productivity employees, means for the natural language processing model to analyze the employee dialogue and determine what is good and what is bad, means for the natural language processing model to correct the employee dialogue, means for recognizing customer emotions using the natural language processing model incorporating an emotion engine and suggesting to the employee a response based on the emotion, and means installed in the smart glasses to suggest a response to the employee in real time. This enables the employee to instantly recognize the customer's emotions and take an appropriate response in real time.

[1432] A "highly productive employee" is an employee who is highly efficient and effective in their interactions with customers.

[1433] "Low productivity employees" are employees who are inefficient and ineffective in their interactions with customers.

[1434] "Dialogue" is the verbal exchange that takes place between employees and customers.

[1435] A "natural language processing model" is an artificial intelligence technology for understanding and analyzing human language.

[1436] "Analysis" is the process of examining and evaluating the content of a dialogue in detail.

[1437] "Judgment" is the act of evaluating the quality of a dialogue and drawing a conclusion.

[1438] "Correction" is the act of correcting and improving the content of a dialogue.

[1439] An "emotion engine" is a technology for recognizing and analyzing customer emotions from conversations.

[1440] "Suggestion" is the act of suggesting how to respond to an employee based on perceived emotions.

[1441] "Smart glasses" are wearable devices that have the ability to display information and capture audio and video.

[1442] "Real-time" refers to immediate processing or response without delay.

[1443] This invention is a system that recognizes customer emotions in real time when employees interact with customers in a physical store and suggests appropriate responses. This system combines a natural language processing model and an emotion engine and is implemented using smart glasses.

[1444] System configuration

[1445] The system consists of the following main components:

[1446] 1. Natural language processing model: Learns the interactions between high- and low-productivity employees and determines whether the interaction is good or bad.

[1447] 2. Emotion engine: Recognizes emotions from customer interactions and suggests responses based on those emotions.

[1448] 3. Smart glasses: worn by employees, they display conversation content and suggested responses in real time.

[1449] Program processing

[1450] The server implements a natural language processing model and emotion engine using Python, spaCy, and the Transformers library, while the smart glasses are powered by wearable devices such as Google Glass.

[1451] 1. Data capture: The microphone in the smart glasses captures interactions between employees and customers.

[1452] 2. Data analysis: The captured dialogue data is sent to a server and analyzed using a natural language processing model.

[1453] 3. Emotion Recognition: The emotion engine recognizes customer emotions from conversation data.

[1454] 4. Response Suggestion: Suggest appropriate responses to employees based on perceived emotions.

[1455] 5. Real-time display: The suggested responses are displayed in real time on the smart glasses display.

[1456] Specific examples

[1457] For example, if a customer says, "I'm unhappy with the service I received recently," the emotion engine recognizes "unhappy" and suggests to an employee, "Sorry, sir. What is the problem?"

[1458] Prompt Sentence Examples

[1459] Customer: I'm unhappy with your recent service.

[1460] Employee: Sorry, sir. What is the problem?

[1461] This system allows employees to instantly recognize customer emotions and respond appropriately in real time, which is expected to improve customer satisfaction.

[1462] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1463] Step 1:

[1464] The microphone in the smart glasses captures the conversation between the employee and the customer. The input is the voice data of the employee and the customer, and the output is the captured voice data. The smart glasses transmit the voice data to the server in real time.

[1465] Step 2:

[1466] The server converts the received voice data into text data. The input is the captured voice data, and the output is the converted text data. It converts the voice data into text using speech recognition software (e.g., Google Speech-to-Text API).

[1467] Step 3:

[1468] The server analyzes the text data using a natural language processing model. The input is the converted text data, and the output is the analysis result. A natural language processing model (e.g., spaCy) is used to extract specific patterns and phrases from the text data and determine whether the dialogue is good or bad.

[1469] Step 4:

[1470] The server uses an emotion engine to recognize customer emotions from the analysis results. The input is the analysis results, and the output is the recognized emotion data. The emotion engine (e.g., emotion recognition model from the Transformers library) is used to extract customer emotions from text data.

[1471] Step 5:

[1472] The server proposes an appropriate response based on the recognized emotion data. The input is the recognized emotion data, and the output is the response proposal data. The server generates a specific response method for the employee based on a predefined response pattern.

[1473] Step 6:

[1474] The smart glasses display the response suggestion data received from the server to the employee in real time. The input is the response suggestion data, and the output is the response suggestion displayed on the smart glasses' display. The smart glasses visually display the response suggestion so that the employee can immediately confirm it.

[1475] Step 7:

[1476] The employee responds to the customer based on the response suggestions displayed on the smart glasses. The input is the response suggestions displayed on the smart glasses, and the output is the employee's specific response behavior. The employee then carries out the suggested response method and responds in accordance with the customer's feelings.

[1477] Example 2

[1478] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1479] Conventional crew conversation analysis systems have difficulty analyzing crew conversation content in real time and providing appropriate feedback. Furthermore, they lack the functionality to grasp users' emotions in real time and instruct crews on how to respond accordingly, making it difficult to improve customer satisfaction. This has prevented crew skills and the quality of customer service from being fully improved.

[1480] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1481] In this invention, the server includes means for receiving crew chat content in real time, means for transmitting the chat content to a natural language processing model to detect inappropriate expressions and provide correction instructions, and means for transmitting the chat content to a sentiment analysis engine to analyze the user's emotions and provide response instructions. This makes it possible to analyze the crew chat content in real time and provide appropriate feedback. Furthermore, by understanding the user's emotions in real time and instructing the crew to respond accordingly, customer satisfaction can be improved.

[1482] "Crew" refers to the person in charge of interacting with customers.

[1483] "Natural language processing model" refers to artificial intelligence technology that analyzes the content of crew conversations and provides appropriate feedback.

[1484] "Talk content" refers to the words and phrases used by crew members in their interactions with customers.

[1485] "Real-time" means that processing occurs at the exact moment the crew is interacting with the customer.

[1486] "Inappropriate language" refers to words or phrases that may be offensive or misleading to customers.

[1487] "Correction instructions" refer to instructions for changing inappropriate expressions to appropriate expressions.

[1488] An "emotion analysis engine" refers to technology that analyzes user emotions from chat content and provides the results.

[1489] "User" refers to a customer who interacts with a crew member.

[1490] "Response instructions" refers to instructing the crew to respond appropriately according to the user's emotions.

[1491] "Feedback" refers to providing an evaluation of the crew's talk content and suggestions for improvement.

[1492] MODE FOR CARRYING OUT THE INVENTION

[1493] The present invention is a system that analyzes the content of conversations between crew members and customers in real time and provides appropriate feedback. Specific embodiments of this system will be described below.

[1494] Hardware and software used

[1495] The server plays a central role in receiving and processing the crew's conversation content in real time. The server is a computer system equipped with a high-performance processor and sufficient memory. The terminal is the device through which the crew communicates with customers, and is typically a PC or tablet. The user is the customer who communicates with the crew.

[1496] The software uses a common generative AI model for natural language processing, specifically OpenAI's GPT-4, and IBM Watson's Tone Analyzer for sentiment analysis.

[1497] Program processing

[1498] The server receives the chat content entered by the crew members using their terminals in real time. The received chat content is first sent to a natural language processing model. The natural language processing model analyzes the chat content and determines whether it contains inappropriate language. If inappropriate language is detected, the server provides instructions to the crew members to correct it to appropriate language.

[1499] At the same time, the server also sends the chat content to a sentiment analysis engine, which extracts patterns and phrases from the chat that reflect the user's emotions. If the user uses expressions that indicate joy or satisfaction, the sentiment analysis engine detects this and the server instructs the user to take action to maintain positive emotions, such as expressing gratitude to the crew.

[1500] Specific examples

[1501] For example, if a crew member says to a customer, "This product might not look good on you," the natural language processing model can provide a corrective instruction such as, "I think this product would look great on you."

[1502] Or, if a user says, "This service is absolutely amazing!", the sentiment analysis engine will detect this and instruct the server to reply, "Thank you! Your satisfaction is our greatest joy."

[1503] Prompt Sentence Examples

[1504] "If a crew member uses inappropriate language with a customer, provide instructions to correct it to appropriate language."

[1505] "If the user uses expressions that indicate joy or satisfaction, instruct the crew member to express gratitude."

[1506] This system will enable crew members to communicate with customers more appropriately and effectively, which is expected to improve customer satisfaction.

[1507] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1508] Step 1:

[1509] The server receives the talk content entered by the crew using the terminal in real time.

[1510] Input: What the crew member types into the terminal (e.g., "This product may not suit you").

[1511] Output: The received chat content is stored as internal data.

[1512] Specific operation: The crew member inputs the content of the conversation into the terminal, and the terminal sends the data to the server. The server receives this data and proceeds to the next processing step.

[1513] Step 2:

[1514] The server sends the received talk content to a natural language processing model.

[1515] Input: The message received by the server

[1516] Output: The conversation sent to the natural language processing model

[1517] Specific operation: The server sends the received chat content to a natural language processing model, which analyzes the chat content and determines whether it contains inappropriate language.

[1518] Step 3:

[1519] A natural language processing model detects inappropriate language and generates correction instructions.

[1520] Input: The conversation sent to the natural language processing model

[1521] Output: Correction instructions (e.g., "I think this product looks great on you.")

[1522] Specific operation: The natural language processing model analyzes the content of the conversation and determines that the expression "It might not suit you" is inappropriate. The model then generates instructions to correct it to "I think it suits you very much" and returns them to the server.

[1523] Step 4:

[1524] The server sends correction instructions from the natural language processing model to the crew's terminal.

[1525] Input: Correction instructions from a natural language processing model

[1526] Output: Corrective instructions displayed on the crew's terminal

[1527] Specific operation: The server sends the correction instructions received from the natural language processing model to the crew's device, where the correction suggestions are displayed, and the crew member corrects the talk based on them.

[1528] Step 5:

[1529] The server sends the received chat content to a sentiment analysis engine.

[1530] Input: The message received by the server

[1531] Output: The conversation content sent to the sentiment analysis engine

[1532] Specific operation: The server sends the same chat content to the sentiment analysis engine and requests sentiment analysis.

[1533] Step 6:

[1534] The sentiment analysis engine analyzes the user's sentiment and returns the results to the server.

[1535] Input: The conversation content sent to the sentiment analysis engine

[1536] Output: User sentiment analysis results (e.g., positive sentiment)

[1537] How it works: The sentiment analysis engine analyzes the content of conversations and extracts patterns and phrases that reflect the user's emotions. If the user says "great," it detects positive emotions and returns the results to the server.

[1538] Step 7:

[1539] The server provides response instructions to the crew based on the sentiment analysis results.

[1540] Input: Sentiment analysis results from the sentiment analysis engine

[1541] Output: Instructions to be displayed on the crew's device (e.g. "Thank you! Your satisfaction is our greatest joy.")

[1542] Specific operation: Based on the emotion analysis results, the server generates response instructions for the crew and sends them to the crew's terminal. The response instructions are displayed on the crew's terminal, and the crew responds to the user based on the instructions.

[1543] In this way, crew members can interact with customers more appropriately and effectively.

[1544] (Application example 2)

[1545] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1546] With conventional crew conversation analysis systems, it was difficult to analyze crew conversation content in real time and provide appropriate advice. It was also difficult to grasp changes in customer emotions in real time and suggest appropriate responses to the crew. As a result, improvements in crew skills and customer satisfaction were not fully achieved.

[1547] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1548] In this invention, the server includes: means for including a natural language processing model for learning the talk of high-productivity crew members and low-productivity crew members; means for the natural language processing model to analyze the crew members' talk and determine what is good and what is bad; means for the natural language processing model to correct the crew members' talk; means for the natural language processing model to analyze the crew members' talk in real time and provide corrections and advice as needed; means for an emotion engine to extract patterns and phrases that reflect the user's emotions from the crew members' talk; means for the emotion engine to grasp changes in the user's emotions in real time and suggest corresponding responses to the crew members; and means for including an application to be installed on the smart glasses. This makes it possible to analyze the content of the crew members' talk in real time and provide appropriate advice, and further to grasp changes in customer emotions in real time and suggest corresponding responses to the crew members.

[1549] A "highly productive crew" is a staff member who is able to achieve high efficiency and results in customer service.

[1550] A "low-productivity crew" is a staff member who is inefficient and unproductive in dealing with customers.

[1551] A "natural language processing model" is an algorithm or machine learning model that analyzes text data and understands its meaning and context.

[1552] "Talk" refers to the words and expressions used by crew members when interacting with customers.

[1553] "Real-time analysis" refers to the process of instantly analyzing the content of crew conversations and providing the results immediately.

[1554] "Correction" refers to correcting the content of the crew's talk and changing it to appropriate expression.

[1555] "Advice" means instructions or guidance given to the crew.

[1556] An "emotion engine" is an algorithm or machine learning model that analyzes user emotions from text data and detects changes in those emotions.

[1557] A "pattern or phrase" is a combination of words or a sentence used to express a particular meaning or emotion.

[1558] "Smart glasses" are glasses-type devices that have built-in displays and sensors and have the ability to display information and collect data.

[1559] A system for implementing this invention is configured as follows: The server includes a natural language processing model for learning the talk of high-productivity and low-productivity crew members. The natural language processing model analyzes the crew members' talk and determines what is good and what is bad. The natural language processing model also analyzes the crew members' talk in real time and provides corrections and advice as needed. Furthermore, an emotion engine extracts patterns and phrases from the crew members' talk that reflect the user's emotions, grasps changes in the user's emotions in real time, and suggests appropriate responses to the crew members. These functions are provided through an application installed on the smart glasses.

[1560] The server uses a Python program to implement a natural language processing model and emotion engine. The natural language processing model uses the spacy library to analyze Japanese text. The emotion engine uses the emotion analysis model from the transformers library. The content of the crew's conversations is sent to the server in real time via smart glasses and is analyzed immediately.

[1561] Specifically, when a crew member talks with a customer, they wear smart glasses and input their conversation content. The server analyzes the input conversation content using a natural language processing model, detects and corrects inappropriate expressions, and uses an emotion engine to analyze the user's emotions and suggests responses to the crew member based on changes in emotions. This allows the crew member to respond appropriately in real time.

[1562] For example, if a crew member says, "Customer, this is inappropriate expression 1. What do you think?", the server analyzes the conversation and corrects "inappropriate expression 1" to "appropriate expression." If the server determines that the user's emotion is neutral, it provides the crew member with the advice to "continue responding normally."

[1563] An example of a prompt is as follows:

[1564] "Customer, this is an inappropriate expression, what do you think?"

[1565] In this way, crew members can respond appropriately in real time, which is expected to improve customer satisfaction.

[1566] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1567] Step 1:

[1568] The crew puts on the smart glasses and begins interacting with the customer.

[1569] Input: Crew talk

[1570] Output: Talk content input into smart glasses

[1571] Specific operation: The crew inputs the content of the talk through the microphone on the smart glasses.

[1572] Step 2:

[1573] The smart glasses transmit the inputted talk content to the server.

[1574] Input: Talk content entered into the smart glasses

[1575] Output: The chat content sent to the server

[1576] Specific operation: The smart glasses send the conversation content to the server via Wi-Fi or Bluetooth.

[1577] Step 3:

[1578] The server receives the content of the conversation and analyzes it using a natural language processing model.

[1579] Input: The chat content sent to the server

[1580] Output: Parsed chat content

[1581] Specific operation: The server uses the spacy library to analyze the chat content and detect inappropriate expressions.

[1582] Step 4:

[1583] The server corrects inappropriate expressions and converts them into appropriate expressions.

[1584] Input: Parsed chat content

[1585] Output: Modified chat content

[1586] Specific operation: The server replaces inappropriate expressions with appropriate expressions. For example, it corrects "inappropriate expression 1" to "appropriate expression."

[1587] Step 5:

[1588] The server analyzes the user's emotions using an emotion engine.

[1589] Input: Edited chat content

[1590] Output: User sentiment analysis results

[1591] What it does: The server uses a sentiment analysis model from the transformers library to classify the user's sentiment as positive, negative, or neutral.

[1592] Step 6:

[1593] The server suggests responses to the crew based on the user's emotions.

[1594] Input: User sentiment analysis results

[1595] Output: Advice for crew

[1596] Specific operation: If the user's emotion is positive, the server generates advice such as "Express gratitude," if negative, "Respond politely," and if neutral, "Continue responding normally."

[1597] Step 7:

[1598] The server sends the modified talk content and advice to the smart glasses.

[1599] Input: Revised talk, advice to crew

[1600] Output: Corrective speech and advice sent to smart glasses

[1601] Specific operation: The server sends the corrected talk content and advice to the smart glasses via Wi-Fi or Bluetooth.

[1602] Step 8:

[1603] Smart glasses display modified talk and advice to the crew.

[1604] Input: Corrected speech and advice sent to smart glasses

[1605] Output: Corrective talk and advice displayed to the crew

[1606] Specific operation: The revised talk content and advice are displayed on the smart glasses display, and the crew confirms it.

[1607] In this way, crews can take appropriate action in real time.

[1608] Example 3

[1609] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1610] Conventional systems for improving crew talking skills do not adequately analyze the content of crew conversations or recognize the user's emotions, making it difficult for crew members to objectively understand their own talking skills and take concrete measures to improve them. Furthermore, there was a lack of feedback on the user's emotions, making it difficult for crew members to understand how they influenced the user's emotions. This led to the issue of ineffective improvement of crew talking skills.

[1611] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1612] In this invention, the server includes means for collecting crew talk data and transmitting it to the server, means for analyzing the user's emotions using an emotion engine for recognizing the user's emotions, and means for generating feedback by integrating areas for strengthening or improving talk skills and the user's emotion recognition results. This allows crew members to objectively understand their own talk skills and take specific measures to improve them. Furthermore, by receiving feedback on the user's emotions, crew members can understand how they affected the user's emotions and use the results to improve their skills.

[1613] A "highly productive crew" is a crew that can achieve high results in their work.

[1614] A "low-productivity crew" refers to a crew member who performs poorly in their work.

[1615] A "natural language processing model" refers to an algorithm or system for analyzing and understanding natural language.

[1616] "Talk data" refers to voice data and text data generated when a crew member converses with a user.

[1617] "Server" refers to a computer system for processing and storing data.

[1618] An "emotion engine" refers to an algorithm or system for analyzing and recognizing a user's emotions.

[1619] "Feedback" refers to information that integrates areas for strengthening and improving the crew's talking skills and the results of user emotion recognition.

[1620] "Talk skills" refers to the communication ability of a crew member when interacting with a user.

[1621] "Areas of improvement" refers to the areas in which the crew's talking skills are particularly strong.

[1622] "Areas for improvement" refers to areas in the crew's speaking skills that need improvement.

[1623] "User emotion" refers to the emotional state that a user exhibits during an interaction.

[1624] "Analysis" refers to the process of examining data in detail and extracting meaning and patterns.

[1625] "Integration" refers to the process of bringing together multiple pieces of information.

[1626] This invention is a system for improving crew members' conversation skills, which uses a natural language processing model and an emotion engine to analyze the content of crew members' conversations and provide feedback. A specific embodiment of this system will be described below.

[1627] Hardware and software used

[1628] Hardware: Servers, terminals

[1629] Software: Natural language processing models (e.g., GPT-4), emotion engines (e.g., IBM Watson Tone Analyzer)

[1630] System configuration

[1631] 1. Device: When the crew interacts with the user, the voice and text data are collected. For example, when the crew says, "This product is very high performance," the voice is captured by the device's microphone.

[1632] 2. Terminal: The collected data is converted into text format and sent to the server. In the case of voice data, it is converted into text using voice recognition software (e.g., Google Speech-to-Text).

[1633] 3. Server: Inputs the received text data into a natural language processing model (e.g., GPT-4). For example, it analyzes the text "This product is very high performance."

[1634] 4. Server: A natural language processing model analyzes the conversation and extracts areas for skill improvement and strengthening. For example, it generates feedback such as, "You need to provide more details about the specific features of your product."

[1635] 5. Server: Use an emotion engine (e.g. IBM Watson Tone Analyzer) to recognize the user's emotion. For example, if the user says "But it's expensive," input that text into the emotion engine.

[1636] 6. Server: The emotion engine analyzes the user's emotions and recognizes emotions such as dissatisfaction and joy. For example, it gives a result such as "The user is dissatisfied with the price."

[1637] 7. Server: Integrates the points for strengthening and improving the talking skill and the results of user emotion recognition to generate feedback. For example, it creates specific advice such as, "To resolve the user's dissatisfaction, it would be a good idea to also explain the cost-effectiveness of the product."

[1638] 8. Server: Sends the generated feedback to the device.

[1639] 9. Terminal: The crew receives feedback and uses it to improve their speaking skills. For example, the terminal screen might say, "To resolve the user's complaints, it would be a good idea to also explain the cost-effectiveness of the product."

[1640] Examples and prompts

[1641] Examples:

[1642] Crew talk: "Customer, this product is very high performance and will last a long time."

[1643] User emotion recognition result: Dissatisfaction (e.g., the user is dissatisfied with the price)

[1644] Feedback: "To alleviate customer dissatisfaction, it would be a good idea to explain the cost-effectiveness of the product."

[1645] Example prompt sentence:

[1646] "Analyze the crew's conversations and identify areas for skill strengthening and improvement."

[1647] "Recognize user emotions and analyze how crew talk influences them."

[1648] In this way, the system supports the crew in improving their talking skills. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1649] Step 1:

[1650] Collecting crew talk data

[1651] Device: When the crew interacts with the user, the voice and text data are collected in real time. For example, when the crew says, "This product is very high performance," the voice is captured by the device's microphone.

[1652] Input: Crew voice data

[1653] Output: Text conversion result of audio data

[1654] What it does: Your device's microphone captures your voice and converts it into text using speech recognition software (e.g., Google Speech-to-Text).

[1655] Step 2:

[1656] Sending chat data

[1657] Terminal: Sends collected text data to the server.

[1658] Input: Text data converted by speech recognition software

[1659] Output: Text data sent to the server

[1660] Specific operation: The terminal sends text data to the server via the network.

[1661] Step 3:

[1662] Talk data analysis

[1663] Server: Inputs the received text data into a natural language processing model (e.g., GPT-4). For example, it analyzes the text "This product is very high performance."

[1664] Input: Text data sent to the server

[1665] Output: Analysis result of the talk content

[1666] What it does: Natural language processing models analyze text data and identify areas for skill enhancement and improvement.

[1667] Step 4:

[1668] emotion recognition

[1669] Server: Use an emotion engine (e.g. IBM Watson Tone Analyzer) to recognize the user's emotion. For example, if the user says "But it's expensive," input that text into the emotion engine.

[1670] Input: User's text data

[1671] Output: User emotion recognition result

[1672] Specific operation: The emotion engine analyzes text data and recognizes the user's emotions.

[1673] Step 5:

[1674] Generate feedback

[1675] Server: Integrates the results of the user's emotion recognition and the areas for improvement in the talk skill to generate feedback. For example, it creates specific advice such as, "To resolve the user's dissatisfaction, it would be a good idea to also explain the cost-effectiveness of the product."

[1676] Input: Analysis results of chat content, user emotion recognition results

[1677] Output: The generated feedback

[1678] Specific operation: The server integrates the analysis results and emotion recognition results and generates feedback.

[1679] Step 6:

[1680] Providing Feedback

[1681] Server: Sends the generated feedback to the device.

[1682] Input: Generated feedback

[1683] Output: Feedback sent to the device

[1684] Specific operation: The server sends feedback to the device via the network.

[1685] Step 7:

[1686] Receiving and viewing feedback

[1687] Terminal: Crew members receive feedback to help them improve their speaking skills. For example, the terminal screen displays a message saying, "To resolve user complaints, it would be a good idea to also explain the cost-effectiveness of the product."

[1688] Input: Feedback sent by the server

[1689] Output: Feedback displayed to the crew

[1690] What happens: The device receives the feedback and displays it on the screen.

[1691] (Application example 3)

[1692] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1693] Conventional systems for improving crew conversation skills are limited to analyzing and correcting the content of crew conversations, and lack the ability to recognize and provide feedback on customer emotions in real time. This makes it difficult for crew members to immediately understand how their conversations have affected customers, resulting in a decrease in the efficiency of skill improvement.

[1694] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1695] In this invention, the server includes a natural language processing model for learning crew talk, means for the natural language processing model to analyze the crew talk and determine what is good and what is bad, means for the natural language processing model to correct the crew talk, means for feeding back the correction results of the crew talk to the crew and promoting the improvement of the crew's skills, an emotion recognition engine for recognizing customer emotions, means for the emotion recognition engine to analyze customer emotions and feed back the results to the crew, and means for displaying the feedback in real time on a smart device worn by the crew, thereby enabling the crew to instantly understand how their talk affected the customer and efficiently improve their skills.

[1696] A "natural language processing model" is a machine learning algorithm that analyzes crew talk and determines what is good and what is bad.

[1697] "Crew" refers to employees who deal with customers in physical stores, etc.

[1698] "Talk" refers to what the crew says when interacting with customers.

[1699] "Correction" is the act of correcting the content of the crew's talk and pointing out areas for improvement and strengthening.

[1700] "Feedback" refers to the act of communicating the results of the analysis of the talk and the customer's emotion recognition to the crew.

[1701] An "emotion recognition engine" is an algorithm that analyzes a customer's facial expressions and tone of voice to recognize their emotions.

[1702] "Smart devices" refer to electronic devices worn by crew members that can display information in real time, including smart glasses and head-mounted displays.

[1703] "Real-time" means that information is provided at the exact moment the crew is interacting with the customer.

[1704] A system for implementing this invention includes a natural language processing model and an emotion recognition engine to improve crew members' speaking skills, and provides real-time feedback via smart devices worn by the crew members.

[1705] Hardware and software used

[1706] Hardware: Smart glasses, camera

[1707] Software: OpenCV (camera image acquisition and processing), Transformers (natural language processing model), emotion recognition engine

[1708] System processing overview

[1709] 1. Camera initialization and image acquisition

[1710] The server initializes the camera and captures real-time video footage of the interaction between the crew and the customer.

[1711] 2. Customer Emotion Recognition

[1712] The server detects the customer's face from the video frame and analyzes the customer's emotions using an emotion recognition engine, which analyzes the customer's facial expressions and tone of voice to recognize their emotions.

[1713] 3. Crew talk analysis

[1714] The server inputs the crew's conversation content into a natural language processing model and obtains the correction results. The natural language processing model analyzes the crew's conversation and determines what is good and what is bad.

[1715] 4. Providing Feedback

[1716] The server then provides feedback to the crew on the results of the conversation analysis and customer emotion recognition, which is displayed in real time on smart glasses worn by the crew.

[1717] Specific examples

[1718] For example, if a crew member asks, "Sir, what product are you looking for today?", the natural language processing model analyzes this speech and determines whether it is an appropriate phrase. At the same time, if the customer replies with a smile, "Thank you," the emotion recognition engine recognizes the customer's emotion as "happiness." These results are displayed in real time on the crew member's smart glasses, allowing the crew member to instantly understand how their speech affected the customer.

[1719] Prompt Sentence Examples

[1720] "Customer, what product are you looking for today?"

[1721] "Let me know if you need any help."

[1722] In this way, crew members receive real-time feedback and can efficiently improve their customer service skills.

[1723] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1724] Step 1:

[1725] The server initializes the camera and captures the interaction between the crew and the customer in real time. The input is the camera image and the output is the video frame. Specifically, OpenCV is used to capture the video stream from the camera and process each frame.

[1726] Step 2:

[1727] The server detects the customer's face from the video frame and analyzes the customer's emotion using an emotion recognition engine. The input is the video frame and the output is the customer's emotion data. Specifically, the emotion recognition engine detects the face in the frame and recognizes the emotion by analyzing facial expressions and tone of voice.

[1728] Step 3:

[1729] The server inputs the crew's talk content into a natural language processing model and obtains the correction results of the talk. The input is the crew's talk text, and the output is the correction results of the talk. Specifically, it uses Transformers to analyze the talk text and determine what is good and what is bad.

[1730] Step 4:

[1731] The server feeds back the results of the conversation analysis and the customer's emotion recognition to the crew. The input is the conversation correction results and the customer's emotion data, and the output is feedback information. Specifically, the server generates data to display these results in real time on the smart glasses and provides it to the crew.

[1732] Step 5:

[1733] Crew members receive real-time feedback through the smart glasses to improve their customer service skills. The input is feedback information, and the output is improvements to the crew members' behavior. Specifically, the feedback information is displayed on the smart glasses' display, and the crew members use it to improve their conversations and responses.

[1734] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1735] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1736] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1737] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1738] [Third embodiment]

[1739] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1740] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1741] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1742] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1743] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1744] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1745] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1746] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1747] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1748] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1749] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1750] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1751] "Example 1"

[1752] In one embodiment of the present invention, the natural language processing model learns the conversations of highly productive and less productive crew members. Specifically, data such as phrases, expressions, and conversation flow used by crew members during conversations with customers is collected and used as training data. This training data serves as a criterion for judging the quality of a crew member's conversation.

[1753] "Example 2"

[1754] Furthermore, the natural language processing model corrects the crew's conversations. Specifically, when the crew interacts with a customer, the content of the conversation is analyzed in real time and corrections and advice are provided as needed. For example, if the crew uses an inappropriate expression while speaking to a customer, the natural language processing model detects the expression and provides instructions to the crew to correct it to an appropriate expression.

[1755] "Example 3"

[1756] The natural language processing model also provides feedback to help crew members improve their skills. Specifically, based on the results of the corrections to the crew's conversations, it clarifies areas for improvement and strengthens the crew's conversation skills, and provides this feedback to the crew. This allows the crew to objectively understand their own conversation skills and take concrete steps to improve them.

[1757] The processing flow of each embodiment will be described below.

[1758] "Example 1"

[1759] Step 1: The natural language processing model learns the conversations of high-productivity and low-productivity crew members. Specifically, it captures data such as phrases, expressions, and conversation flow used by crew members during conversations with customers.

[1760] Step 2: The captured data is used as training data to train a natural language processing model. Through this training, the model learns the criteria for judging whether a crew member's talk is good or bad.

[1761] "Example 2"

[1762] Step 1: The natural language processing model analyzes crew conversations in real time. Specifically, it captures the conversations that crew members make with customers in real time.

[1763] Step 2: Analyze the captured conversation and provide corrections or advice as needed. For example, if a crew member uses inappropriate language when speaking to a customer, the natural language processing model will detect that expression.

[1764] Step 3: Provide the crew with instructions to correct the detected inappropriate expressions into appropriate expressions. This allows the crew to improve their own talk in real time. "Example 3"

[1765] Step 1: Based on the results of the crew's talk corrections, the natural language processing model identifies areas for strengthening and improving the crew's talking skills.

[1766] Step 2: Provide feedback to the crew on the identified areas for improvement. This allows the crew to objectively understand their own speaking skills and take concrete steps to improve.

[1767] Example 1

[1768] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1769] In conventional crew dialogue evaluation systems, the criteria for judging the quality of dialogue were vague, making it difficult to effectively promote crew skill improvement. Furthermore, the steps of voice data text conversion, data preprocessing, feature extraction, model learning, evaluation, and feedback were not integrated, making efficient system operation difficult.

[1770] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1771] In this invention, the server includes means for converting voice data into text data, means for preprocessing the text data, means for extracting features from the preprocessed data, means for training a model using the extracted features, means for evaluating the performance of the trained model, means for improving the model based on the evaluation results, means for analyzing the crew's dialogue and determining what is good and what is bad, and means for correcting the crew's dialogue. This makes it possible to accurately determine the quality of the crew's dialogue and provide effective feedback.

[1772] A "highly productive crew" is a crew that can achieve high results in interactions with customers.

[1773] A "low-productivity crew" refers to a crew member who performs poorly in interactions with customers.

[1774] "Natural language processing model" refers to an algorithm or machine learning model for understanding and processing natural language.

[1775] "Dialogue" refers to the verbal exchange between the crew and the customer.

[1776] "Audio Data" refers to audio files that record the conversation between the crew and the customer.

[1777] "Text data" refers to data obtained by converting voice data into character information.

[1778] "Preprocessing" refers to the process of removing unnecessary information from text data and converting it into a format that is easy to analyze.

[1779] "Feature extraction" refers to the process of extracting useful information and patterns from text data.

[1780] "Model learning" refers to the process of training a machine learning model using the extracted features.

[1781] "Evaluation" refers to the process of measuring the performance of a trained model and confirming its accuracy and effectiveness.

[1782] "Feedback" refers to information provided to the crew members to promote skill improvement through evaluation results of the model and corrections to the dialogue.

[1783] The present invention is a system including a natural language processing model for learning the dialogue between a high-productivity crew and a low-productivity crew. A specific embodiment of this system will be described below.

[1784] System Overview

[1785] The user generates a program that uses a natural language processing model to learn the conversations between high-productivity and low-productivity crew members. This program takes in data such as phrases, expressions, and the flow of conversations used by crew members when interacting with customers, and uses this data as learning data.

[1786] Hardware and software used

[1787] Server: Collects data, preprocesses it, extracts features, trains it, evaluates it, and provides feedback.

[1788] Speech recognition software: Converts voice data into text data (e.g., Google Speech-to-Text API).

[1789] Natural language processing libraries: Preprocess text data (e.g., NLTK, spaCy).

[1790] Machine learning libraries: for feature extraction and model training (e.g., scikit-learn, TensorFlow, PyTorch).

[1791] Data processing and calculation

[1792] 1. Data Collection:

[1793] The server records the phrases and expressions used by the crew members while interacting with customers, as well as the flow of the conversation, and uses voice recognition software to convert the voice data into text data.

[1794] 2. Data Preprocessing:

[1795] The server preprocesses the collected text data, specifically by removing noise, tokenizing, and removing stop words.

[1796] 3. Feature Extraction:

[1797] The server extracts features from the preprocessed data, such as frequent phrases, sentiment analysis, and dialogue flow.

[1798] 4. Model training:

[1799] The server uses the extracted features to train a natural language processing model. The model is trained using the dialogue data of the high-productivity crew and the low-productivity crew as labeled data.

[1800] 5. Rating and Feedback:

[1801] The server evaluates the performance of the trained model using metrics such as precision, recall, and F1 score, improves the model based on the evaluation results, and provides feedback to the crew.

[1802] Specific examples

[1803] Example dialogue from a high-productivity crew: "Customers, this product is very popular. This feature is especially popular."

[1804] Example dialogue from a low-productivity crew: "Well, this product is okay. Nothing special."

[1805] Prompt Sentence Examples

[1806] "Generate a program to train a natural language processing model using the dialogue data of high-productivity and low-productivity crews. Specifically, the process will include the steps of collection, preprocessing, feature extraction, model training, and evaluation."

[1807] In this way, users can build a system that uses a natural language processing model to learn criteria for judging whether a crew member's dialogue is good or bad.

[1808] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1809] Step 1: Data collection

[1810] Input: Voice data of conversation between crew and customer

[1811] What it does: The server records the phrases and expressions used by the crew members while interacting with the customer, as well as the flow of the conversation. It then converts the voice data into text using speech recognition software (e.g., Google Speech-to-Text API).

[1812] Output: Text data

[1813] Step 2: Data Preprocessing

[1814] Input: Collected text data

[1815] Specific operation: The server preprocesses the collected text data, specifically removing noise, tokenizing, and removing stop words. This process uses natural language processing libraries (e.g., NLTK, spaCy).

[1816] Output: Preprocessed text data

[1817] Step 3: Feature extraction

[1818] Input: Preprocessed text data

[1819] Specific operation: The server extracts features from the preprocessed data, such as frequently used phrases, sentiment analysis, and dialogue flow. This process uses machine learning libraries (e.g., scikit-learn, TensorFlow).

[1820] Output: Extracted feature data

[1821] Step 4: Model training

[1822] Input: Extracted feature data and label data (interaction data of high-productivity crew and low-productivity crew)

[1823] Specific operation: The server uses the extracted features to train a natural language processing model. The conversation data of the high-productivity crew and the low-productivity crew are used as labeled data to train the model. This process uses a deep learning framework (e.g., PyTorch, TensorFlow).

[1824] Output: Trained model

[1825] Step 5: Rating and feedback

[1826] Input: trained model and test data

[1827] Specific operation: The server evaluates the performance of the trained model using metrics such as precision, recall, and F1 score, improves the model based on the evaluation results, and provides feedback to the crew.

[1828] Output: Evaluation results and feedback information

[1829] In this way, users can build a system that uses a natural language processing model to learn criteria for judging whether a crew member's dialogue is good or bad.

[1830] (Application example 1)

[1831] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1832] In conventional customer service work, there was a problem in which variations in crew conversation skills affected customer satisfaction. In particular, the conversations of low-productivity crew were often ineffective in communicating with customers, which led to lower sales and customer satisfaction. Another issue was the lack of feedback and training for crew to improve their conversation skills. To solve these problems, a system was needed that would help crew use appropriate phrases and expressions in real time and promote the improvement of their conversation skills.

[1833] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1834] In this invention, the server includes a natural language processing model for learning the talk of high-productivity crew members and low-productivity crew members, means for the natural language processing model to analyze the crew members' talk and determine what is good and what is bad, means for the natural language processing model to correct the crew members' talk, means for suggesting appropriate phrases and expressions in real time when the crew members interact with customers, means for evaluating the crew members' talk and providing feedback, and means for the crew members to use during training to acquire talking skills, thereby enabling the crew members to communicate effectively in interactions with customers, improving customer satisfaction and increasing sales.

[1835] A "highly productive crew" is an employee who has excellent communication skills when interacting with customers and is highly efficient at work.

[1836] "Low-productivity crew" refers to employees who lack communication skills when interacting with customers and have low work efficiency.

[1837] A "natural language processing model" is a model that uses algorithms and techniques to understand and generate human language.

[1838] "Talk" refers to the words, phrases, and expressions used by crew members when interacting with customers.

[1839] "Analysis" refers to the process of capturing crew talk as data and evaluating its content.

[1840] "Judgment" refers to the process of evaluating the quality of a crew member's talk based on evaluation criteria.

[1841] "Correction" is the process of correcting the crew's talk and changing it to more appropriate expressions.

[1842] "Real-time" means responding instantly to the moment the crew is interacting with the customer.

[1843] "Phrases" are short words or expressions that crew members use when interacting with customers.

[1844] "Evaluation" is the process of measuring the effectiveness and appropriateness of the crew's talk based on its content.

[1845] "Feedback" is the process of communicating the results of the crew's talk evaluation and pointing out areas for improvement and good points.

[1846] "Training" refers to the learning and practice that crew members undertake to improve their skills.

[1847] "Skills" refer to the techniques and abilities that crew members use in interacting with customers.

[1848] As an embodiment of the present invention, the following system can be constructed.

[1849] System Overview

[1850] The system suggests appropriate phrases and expressions in real time when the crew interacts with customers, and provides evaluations and feedback on the conversation. The system consists of a server and a terminal including smart glasses.

[1851] Hardware and software used

[1852] Hardware:

[1853] Smart glasses: equipped with a display and microphone, worn by the crew.

[1854] Server: A cloud server to host the natural language processing model and process data in real time.

[1855] software:

[1856] spaCy: A natural language processing library.

[1857] Transformers: Hugging Face's Transformers model library.

[1858] GPT-2: A generative AI model.

[1859] Data processing and calculation

[1860] 1. Talk Analysis:

[1861] Voice data from crew members interacting with customers is collected by the microphone in the smart glasses and converted into text data.

[1862] The server analyzes the collected text data using spaCy and evaluates the content of the talk.

[1863] 2. Real-time support:

[1864] The crew's input text is fed into the GPT-2 model to generate appropriate phrases and expressions.

[1865] The generated phrases and expressions are displayed in real time on the smart glasses' display.

[1866] 3. Talk evaluation and feedback:

[1867] The server evaluates the crew's talk and calculates a score.

[1868] The evaluation results are provided as feedback to the crew and used in training mode.

[1869] Specific examples

[1870] When the crew member asks, "Hello, customer. What products are you looking for today?", the smart glasses' display will show, "We have a new collection in stock. Please take a look."

[1871] Prompt Sentence Examples

[1872] Customer: Hello, do you have any recommendations for me today?

[1873] Crew: Hello, customer. What product are you looking for today?

[1874] In this way, the system supports crew conversations in real time, enabling smoother dialogue with customers, and can also encourage crew skill improvement through conversation evaluation and feedback.

[1875] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1876] Step 1:

[1877] The crew member begins a conversation with the customer.

[1878] Input: Crew voice.

[1879] Specific operation: The crew puts on the smart glasses and starts talking to the customer. The microphone in the smart glasses collects the crew's voice.

[1880] Step 2:

[1881] The terminal converts the voice data into text data.

[1882] Input: Crew voice data.

[1883] Data processing: Converting voice data into text using speech recognition software.

[1884] Output: Crew text data.

[1885] Specific operation: The built-in microphone of the smart glasses collects voice data and converts it into text data in real time.

[1886] Step 3:

[1887] The server analyzes the text data and generates appropriate phrases and expressions.

[1888] Input: Crew text data.

[1889] Data Computation: Analyzes text data using a natural language processing model (spaCy) and generates appropriate phrases and expressions using a generative AI model (GPT-2).

[1890] Output: Suggested phrases or expressions.

[1891] How it works: The server receives the text data, analyzes it, and then uses a generative AI model to generate appropriate phrases and expressions.

[1892] Step 4:

[1893] The device displays the generated phrases and expressions to the crew in real time.

[1894] Input: A suggested phrase or expression.

[1895] Output: A phrase or expression that will be displayed on the smart glasses display.

[1896] What it does: The generated phrases and expressions are displayed in real time on the smart glasses display, allowing the crew member to refer to them as they continue their conversation with the customer.

[1897] Step 5:

[1898] The server rates the crew's talk and provides feedback.

[1899] Input: Crew text data.

[1900] Data computation: Using natural language processing models to evaluate the content of the talk and calculate a score.

[1901] Output: Evaluation results as feedback.

[1902] What it does: The server evaluates the crew's talk, calculates a score, and generates feedback.

[1903] Step 6:

[1904] Crew members receive feedback and improve their skills in training mode.

[1905] Input: Evaluation results as feedback.

[1906] Output: Crew skill improvement.

[1907] What it does: Crew members receive feedback and practice to improve their speaking skills in training mode.

[1908] Example 2

[1909] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1910] Conventional dialogue support systems have the problem that it is difficult for workers to receive appropriate feedback on the spot when they use inappropriate expressions during a conversation with a customer. Additionally, there is a lack of specific advice to improve the quality of the conversation, which means that workers are unable to sufficiently improve their skills.

[1911] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1912] In this invention, the server includes means for analyzing the worker's dialogue in real time and determining what is good and what is bad, means for correcting the worker's dialogue, means for detecting inappropriate expressions and providing the worker with instructions to correct them to appropriate expressions, means for transmitting the generated advice to the worker's terminal, and means for the terminal to display the advice to the worker. This enables the worker to receive appropriate feedback in real time during a dialogue with a customer, and by providing specific advice to improve the quality of the dialogue, the worker's skills can be improved.

[1913] A "highly productive worker" refers to a worker who achieves high efficiency and results in their work.

[1914] "Low productivity workers" refer to workers who are low in efficiency and produce low results in their work.

[1915] A "natural language processing model" refers to an algorithm or machine learning model that understands and generates human language.

[1916] "Dialogue" refers to communication between a worker and a customer.

[1917] "Real-time analysis" refers to analyzing data instantly while a conversation is taking place.

[1918] "Inappropriate language" refers to words or phrases that may be offensive or misleading to customers.

[1919] "Appropriate expressions" refer to words and phrases that leave a favorable impression on customers and convey accurate information.

[1920] "Advice" refers to specific instructions or suggestions for the worker to improve the interaction.

[1921] "Terminal" refers to a computer or mobile device used by a worker.

[1922] "Feedback" refers to evaluations and information on areas for improvement provided to workers.

[1923] "Skill improvement" refers to improving workers' work abilities and communication skills.

[1924] The present invention provides a system for providing appropriate feedback in real time when a worker is interacting with a customer. The system includes a server, a terminal, and a natural language processing model.

[1925] Hardware and software used

[1926] Hardware: Servers, terminals (devices used by workers)

[1927] Software: Natural language processing models (e.g., GPT-4), real-time data analysis tools (e.g., Apache Kafka)

[1928] System Operation

[1929] 1. The user (worker) initiates a conversation with the customer. The worker uses a device (e.g., a tablet or PC) to communicate with the customer. The conversation is recorded on the device via voice input or text input.

[1930] 2. The device sends the conversation content to the server in real time using the WebSocket or HTTP protocol.

[1931] 3. The server inputs the received dialogue into the natural language processing model. The input data is in text format, and is preprocessed as necessary.

[1932] 4. The natural language processing model analyzes the dialogue and identifies inappropriate expressions and areas for improvement, such as "This product may not be suitable for you."

[1933] 5. The server receives the analysis results from the natural language processing model and generates appropriate expressions and advice, such as "You might want to compare this product with other options."

[1934] 6. The server sends the generated advice to the device using the WebSocket or HTTP protocol.

[1935] 7. The terminal displays the received advice to the user (worker) in the form of a pop-up or notification.

[1936] 8. The user (worker) modifies the dialogue based on the displayed advice. For example, they might say, "You might want to compare this product with other options."

[1937] Specific examples

[1938] Scenario: A worker says to a customer, "This product may not be right for you."

[1939] Analysis result of natural language processing model: "Inappropriate expressions were detected. These may create a negative impression on customers."

[1940] Generated advice: "You might want to compare this product with other options."

[1941] Prompt Sentence Examples

[1942] Input prompt: "When a worker says to a customer, 'This product may not be right for you,' please provide advice on how to revise it to an appropriate phrase."

[1943] Sample output: "You might want to compare this product with other options."

[1944] The system provides real-time support to help workers smoothly advance their interactions with customers.

[1945] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1946] Step 1:

[1947] The user (worker) starts a dialogue with the customer.

[1948] Input: The worker uses the terminal to interact with the customer.

[1949] Specific operation: The worker records the conversation content on the terminal through voice input or text input.

[1950] Output: The conversation is logged to the terminal.

[1951] Step 2:

[1952] The terminal transmits the contents of the conversation to the server in real time.

[1953] Input: Dialogue recorded on device.

[1954] Specific operation: The device captures the conversation content in real time and sends it to the server using WebSocket or HTTP protocol.

[1955] Output: The dialogue is sent to the server.

[1956] Step 3:

[1957] The server inputs the received dialogue content into a natural language processing model.

[1958] Input: The conversation sent to the server.

[1959] Specific operation: The server inputs the received dialogue content in text format into the natural language processing model, and performs preprocessing as necessary.

[1960] Output: The dialogue input to the natural language processing model.

[1961] Step 4:

[1962] A natural language processing model analyzes the dialogue and detects inappropriate expressions and areas for improvement.

[1963] Input: The dialogue that is fed into the natural language processing model.

[1964] How it works: The natural language processing model analyzes the conversation and identifies inappropriate expressions and areas for improvement, such as "This product may not be suitable for you."

[1965] Output: Detected inappropriate expressions and areas for improvement.

[1966] Step 5:

[1967] The server receives the analysis results from the natural language processing model and generates appropriate expressions and advice.

[1968] Input: Analysis results from a natural language processing model.

[1969] Specific operation: The server generates appropriate expressions and advice based on the analysis results. For example, it generates advice such as "You may want to compare this product with other options."

[1970] Output: The generated advice.

[1971] Step 6:

[1972] The server sends the generated advice to the terminal.

[1973] Input: The generated advice.

[1974] Specific operation: The server sends the generated advice to the device using WebSocket or HTTP protocol.

[1975] Output: Advice is sent to the terminal.

[1976] Step 7:

[1977] The terminal displays advice to the crew.

[1978] Input: Advice sent to the terminal.

[1979] Specific operation: The terminal displays the received advice to the worker in the form of a pop-up or notification.

[1980] Output: The advice displayed to the worker.

[1981] Step 8:

[1982] The user (operator) corrects the dialogue based on the displayed advice.

[1983] Input: The advice shown.

[1984] Specific behavior: The worker corrects the dialogue based on the displayed advice, for example, saying, "You might want to compare this product with other options."

[1985] Output: The corrected dialogue.

[1986] (Application example 2)

[1987] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1988] In traditional customer service work in brick-and-mortar stores, the content of the conversation between workers has a significant impact on customer satisfaction, but if a worker uses inappropriate language, it is difficult to correct it on the spot. Also, in order to improve the worker's skills, the content of the conversation must be fed back after the fact, making it difficult to improve in real time. This has led to issues such as a decrease in customer satisfaction and delays in improving worker skills.

[1989] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1990] In this invention, the server includes means for analyzing the worker's dialogue and determining what is good and what is bad, means for correcting the worker's dialogue, means for analyzing the worker's dialogue in real time and providing instructions for correcting to appropriate expressions, and means for displaying the corrected expressions on a display device. This makes it possible to correct to appropriate expressions in real time even if the worker uses inappropriate expressions during a dialogue with a customer, thereby improving customer satisfaction and worker skills at the same time.

[1991] A "highly productive worker" is a worker who achieves high efficiency and results in their work.

[1992] A "low-productivity worker" is a worker who is inefficient or produces poor results in their work.

[1993] "Dialogue" refers to verbal exchanges between a worker and a customer.

[1994] A "natural language processing model" is a model that uses artificial intelligence technology to understand, analyze, and generate human language.

[1995] "Analysis" is the process of examining the content of a conversation in detail and determining what is good and what is bad.

[1996] "Correction" refers to correcting the content of a conversation and changing it to appropriate expression.

[1997] "Real-time" refers to processing and analysis occurring at the exact moment the interaction is taking place.

[1998] "Appropriate expression" means using correct language that is not rude or misleading to customers.

[1999] "Instructions" are specific advice or orders given to workers to correct their behavior or words.

[2000] "Display device" means a device for visually displaying modified expressions or instructions.

[2001] A system for implementing the present invention includes smart glasses worn by a worker, a microphone for recognizing voice, a server for executing a natural language processing model, and a display device. A specific embodiment of this system will be described below.

[2002] First, a worker wears smart glasses and interacts with a customer through a microphone. The microphone captures the worker's voice in real time and sends the voice data to a server. The server then converts the voice data into text data using speech recognition software (e.g., the SpeechRecognition library).

[2003] The server then inputs the converted text data into a natural language processing model (e.g., the BERT model from the Transformers library) to analyze the dialogue. The natural language processing model determines whether the dialogue is appropriate and, if it contains inappropriate language, generates instructions to correct it to an appropriate expression.

[2004] The generated correction instructions are displayed in real time on the smart glasses' display device. This allows the worker to check the appropriate expression during the conversation and make immediate corrections. For example, if the worker says, "This product is cheap," the natural language processing model will determine that the expression "cheap" is negative, and the smart glasses will display a correction instruction to read, "This product is reasonably priced."

[2005] This system allows workers to learn appropriate expressions in real time while dealing with customers, thereby improving customer satisfaction and worker skills at the same time.

[2006] An example of a prompt is as follows:

[2007] "If a crew member says to a customer, 'This product is cheap,' generate instructions to correct that."

[2008] In this way, the quality of customer service provided by staff in physical stores can be improved.

[2009] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2010] Step 1:

[2011] The worker wears smart glasses and communicates with the customer through a microphone.

[2012] Input: Worker's voice

[2013] Output: Audio data

[2014] Specific action: The worker initiates a conversation with the customer, and the microphone captures their voice.

[2015] Step 2:

[2016] The audio data captured by the microphone is sent to the server.

[2017] Input: Audio data

[2018] Output: Audio data sent to the server

[2019] Specific operation: The audio data captured by the microphone is sent to the server in real time.

[2020] Step 3:

[2021] The server uses voice recognition software to convert the voice data into text data.

[2022] Input: Audio data

[2023] Output: Text data

[2024] Specific operation: The server uses the SpeechRecognition library to analyze the audio data and generate corresponding text data.

[2025] Step 4:

[2026] The server inputs text data into a natural language processing model and analyzes the content of the conversation.

[2027] Input: Text data

[2028] Output: Analysis results

[2029] Specific operation: The server analyzes the text data using the BERT model from the Transformers library and evaluates the appropriateness of the dialogue content.

[2030] Step 5:

[2031] If the server finds that an inappropriate expression is included, it generates instructions to correct it to an appropriate expression.

[2032] Input: Analysis results

[2033] Output: Correction instructions

[2034] Specific operation: Based on the analysis results of the natural language processing model, the server detects inappropriate expressions and generates instructions to correct them to appropriate expressions.

[2035] Step 6:

[2036] The server transmits the generated correction instructions to the display device of the smart glasses.

[2037] Input: Correction instructions

[2038] Output: Correction instructions displayed on the smart glasses

[2039] Specific operation: The server sends the correction instructions to the display device of the smart glasses so that the worker can check them.

[2040] Step 7:

[2041] The worker checks the correction instructions displayed on the display device of the smart glasses and corrects the content of the dialogue.

[2042] Input: Correction instructions displayed on smart glasses

[2043] Output: Corrected dialogue

[2044] Specific operations: The worker checks the correction instructions displayed on the smart glasses' display and corrects the content of the conversation with the customer to use appropriate expressions.

[2045] Example 3

[2046] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2047] Conventional systems for improving crew conversation skills have had problems such as insufficient analysis of conversation content and feedback, making it difficult for crew members to identify specific areas for improvement. Furthermore, saving conversation content and managing analysis results are often done manually, making it difficult to improve skills efficiently.

[2048] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[2049] In this invention, the server includes a means for the crew to input talk content, a means for the server to receive and store the talk content, a means for the server to send the talk content to a natural language processing model, a means for the natural language processing model to analyze the talk content, a means for the server to receive and store the analysis results, a means for the server to generate feedback, and a means for the crew to receive feedback, which makes it possible to efficiently analyze the talk content of the crew and provide specific feedback.

[2050] A "highly productive crew" is a crew that can achieve high results in their work.

[2051] A "low-productivity crew" refers to a crew member who performs poorly in their work.

[2052] A "natural language processing model" refers to an algorithm or machine learning model that analyzes the content of crew conversations and extracts areas for improvement or strengthening.

[2053] "Talk content" refers to the content of conversations and comments made by crew members while on duty.

[2054] "Server" refers to a computer system that receives and stores crew chat content and manages the analysis results.

[2055] "Feedback" refers to specific advice and areas for improvement provided to the crew based on the results of analysis by the natural language processing model.

[2056] A "prompt sentence" refers to an instruction sentence used when requesting analysis from a natural language processing model.

[2057] "Analysis results" refers to information on areas for enhancement or improvement obtained after the natural language processing model analyzes the content of the talk.

[2058] "Database" refers to an information management system for storing talk content and analysis results.

[2059] "Input form" refers to the interface that crew members use to enter talk content.

[2060] This invention is a system that provides feedback to improve crew members' speaking skills. The system analyzes the content of crew members' conversations using a natural language processing model, and provides feedback by clarifying areas for improvement.

[2061] Hardware and software used

[2062] Hardware: Servers, devices (PCs and smartphones)

[2063] Software: Natural language processing models (e.g., GPT-4), database management systems (e.g., MySQL), web application frameworks (e.g., Django)

[2064] Specific operation of the system

[2065] The user enters the content of the chat.

[2066] The user (crew member) uses the terminal to input the content of the conversation. For example, they might type "Good morning. How can I help you today?" into an input form on a web browser.

[2067] The server receives and stores the chat content.

[2068] The server receives the chat content sent from the device. The server saves the received chat content in a database (e.g., MySQL). When saving, it also records the user ID and timestamp.

[2069] The server sends the conversation content to the natural language processing model

[2070] The server sends the saved chat content to a natural language processing model (e.g., GPT-4). When sending, it generates a prompt and passes it to the model along with the chat content. For example, it generates a prompt like, "Please analyze the following chat content and provide feedback on areas for improvement and strengthening. Chat content: 'Good morning. What can we do for you today?'"

[2071] Natural language processing model analyzes the content of the conversation

[2072] The natural language processing model analyzes the content of the conversation based on the prompt received. As a result of the analysis, it extracts areas for improvement and strengthening. For example, it generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[2073] The server receives and stores the analysis results.

[2074] The server receives the analysis results from the natural language processing model and stores them in a database. When storing them, the server associates the conversation content with the corresponding analysis results.

[2075] The server generates feedback

[2076] The server generates feedback based on the analysis results. The feedback is provided in a format that is easy for the user to understand, and includes specific advice such as "next time, speak in a softer tone" or "use specific friendly words."

[2077] User receives feedback

[2078] The user can then use the device to review the generated feedback, which is displayed in a web browser. Based on the feedback, the user can take specific actions to improve their speaking skills.

[2079] Examples of specific examples and prompts

[2080] Examples:

[2081] The crew enters "how to greet customers."

[2082] The natural language processing model generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[2083] The crew will take steps to improve, such as "next time, speak in a softer tone" or "use specific, friendly words."

[2084] Example prompt sentence:

[2085] Please analyze the following conversation and provide feedback on areas for improvement. Conversation: 'Good morning. How can I help you today?'

[2086] In this way, the system supports the crew in improving their talking skills. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[2087] Step 1:

[2088] The user enters the content of the chat.

[2089] The user (crew member) uses the terminal to input the content of the conversation. For example, they enter "Good morning. How can I help you today?" into the input form. The entered content of the conversation is sent to the system.

[2090] Input: The chat content entered by the user

[2091] Output: The message is sent to the server.

[2092] Step 2:

[2093] The server receives and stores the chat content.

[2094] The server receives the chat content sent from the device and saves it in a database (e.g., MySQL). When saving, it also records the user ID and timestamp.

[2095] Input: Chat content sent from the device

[2096] Output: The chat content is saved in the database.

[2097] Step 3:

[2098] The server sends the conversation content to the natural language processing model

[2099] The server sends the saved chat content to a natural language processing model (e.g., GPT-4). When sending, it generates a prompt and passes it to the model along with the chat content. For example, it generates a prompt like, "Please analyze the following chat content and provide feedback on areas for improvement and strengthening. Chat content: 'Good morning. What can we do for you today?'"

[2100] Input: Chat content stored in the database

[2101] Output: The conversation along with the prompt is sent to the natural language processing model.

[2102] Step 4:

[2103] Natural language processing model analyzes the content of the conversation

[2104] The natural language processing model analyzes the content of the conversation based on the prompt received. As a result of the analysis, it extracts areas for improvement and strengthening. For example, it generates feedback such as "Your greeting tone is too stiff" or "You should use more friendly language."

[2105] Input: Prompt and talk content

[2106] Output: Analysis results (enhancements and improvements)

[2107] Step 5:

[2108] The server receives and stores the analysis results.

[2109] The server receives the analysis results from the natural language processing model and stores them in a database. When storing them, the server associates the conversation content with the corresponding analysis results.

[2110] Input: Analysis results from a natural language processing model

[2111] Output: Analysis results are saved in a database

[2112] Step 6:

[2113] The server generates feedback

[2114] The server generates feedback based on the analysis results. The feedback is provided in a format that is easy for the user to understand, and includes specific advice such as "next time, speak in a softer tone" or "use specific friendly words."

[2115] Input: Analysis results stored in a database

[2116] Output: Feedback is generated

[2117] Step 7:

[2118] User receives feedback

[2119] The user can then use the device to review the generated feedback, which is displayed in a web browser. Based on the feedback, the user can take specific actions to improve their speaking skills.

[2120] Input: Feedback from the server

[2121] Output: User receives and confirms feedback

[2122] (Application example 3)

[2123] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2124] Conventional systems for improving crew conversation skills have difficulty analyzing crew conversation content in real time and providing immediate feedback. Furthermore, the feedback is often not specific, making it unclear how crew members should improve. This leads to delays in crew skill improvement.

[2125] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[2126] In this invention, the server includes a natural language processing model for learning the conversations of high-productivity workers and low-productivity workers, means for the natural language processing model to analyze the workers' conversations and determine what is good and what is bad, means for the natural language processing model to correct the workers' conversations, means for the natural language processing model to analyze the workers' conversations in real time and provide feedback on areas for improvement and strengthening, and means including a display device for displaying the feedback. This allows workers to receive specific feedback in real time, enabling them to improve their skills quickly and effectively.

[2127] A "highly productive worker" is a worker who achieves high efficiency and results in their work.

[2128] A "low-productivity worker" is a worker who is inefficient or produces poor results in their work.

[2129] A "natural language processing model" is a type of artificial intelligence technology for understanding and analyzing human language.

[2130] "Worker conversation" refers to verbal communication that occurs during work.

[2131] "Analysis" is the act of examining data or information in detail to find meaning and patterns.

[2132] "Judgment" is the act of deciding whether something is good or bad or appropriate based on information.

[2133] "Correction" is the act of correcting and improving errors in writing or conversation.

[2134] "Real-time analysis" means processing data instantly and obtaining results immediately.

[2135] "Areas of improvement" are points that can be used to further improve current skills and performance.

[2136] "Areas for improvement" are points in your current skills or performance that need to be corrected or improved.

[2137] "Feedback" is the act of providing evaluation or comment on behavior or results.

[2138] A "display device" is hardware for visually displaying information.

[2139] A system for implementing this invention is configured as follows: The server includes a natural language processing model for learning conversations between high-productivity workers and low-productivity workers. The natural language processing model analyzes the workers' conversations and determines what is good and what is bad. The natural language processing model also corrects the workers' conversations, analyzes them in real time, and provides feedback on areas for improvement and strengthening. The feedback is provided to the workers via a display device.

[2140] Hardware and software used

[2141] Hardware: Smartphones (e.g., iPhone, Android devices), smart glasses (e.g., Google Glass)

[2142] Software: Natural language processing models (e.g., OpenAI GPT-4), speech recognition software (e.g., Google Speech-to-Text, IBM Watson Speech to Text), feedback systems (e.g., custom feedback engines)

[2143] Data processing and calculation

[2144] 1. Voice recognition: The microphone on the device (smartphone or smart glasses) is used to record the worker's speech, which is then converted into text using voice recognition software.

[2145] 2. Natural language processing: The server inputs the converted text into a natural language processing model to analyze the conversation.

[2146] 3. Feedback generation: Based on the analysis results, the server generates enhancements and improvements in a feedback system.

[2147] 4. Real-time feedback: Display feedback on the device display.

[2148] Specific examples

[2149] For example, consider a situation where a staff member in a physical store is explaining a new product to a customer. If the staff member says, "This product is very good. Please try it," the natural language processing model will analyze this conversation and display feedback such as, "It would be good to explain the product's features in more detail. For example, it would be effective to give a specific example such as, 'This product uses the latest technology and is 30% more efficient than previous products.'"

[2150] Prompt Sentence Examples

[2151] Analyze the conversation of a staff member explaining a new product to a customer and generate feedback in the following format:

[2152] Conversation: [Staff conversation]

[2153] Feedback: [Improvements and improvements]

[2154] example:

[2155] Conversation: "This product is great. You should definitely try it."

[2156] Feedback: "It would be helpful to provide a more detailed description of the product's features. For example, you could say, 'This product uses cutting-edge technology and is 30% more efficient than conventional products.'"

[2157] In this way, workers receive real-time, specific feedback, enabling them to improve their skills quickly and effectively.

[2158] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[2159] Step 1:

[2160] Voice Recognition

[2161] The microphone on the device (smartphone or smart glasses) is used to record the worker's conversation. The recorded voice data is input into voice recognition software (e.g., Google Speech-to-Text, IBM Watson Speech to Text) and converted into text data.

[2162] Input: Audio data

[2163] Output: Text data

[2164] Step 2:

[2165] Natural Language Processing

[2166] The server inputs the text data obtained in step 1 into a natural language processing model (e.g., OpenAI GPT-4) to analyze the conversation. The analysis extracts the content, tone, grammar, and specific patterns and phrases of the conversation.

[2167] Input: Text data

[2168] Output: Analysis results (evaluation of conversation content, specific patterns and phrases)

[2169] Step 3:

[2170] Feedback Generation

[2171] The server generates reinforcement and improvement points in the feedback system based on the analysis results obtained in step 2. The generated feedback includes specific advice and improvement measures.

[2172] Input: Analysis results

[2173] Output: Feedback data (enhancements and improvements)

[2174] Step 4:

[2175] Real-time feedback display

[2176] The terminal displays the feedback data generated in step 3 on a display device (such as a smartphone screen or smart glasses display). The worker receives the feedback in real time and can immediately implement improvements.

[2177] Input: Feedback data

[2178] Output: The displayed feedback

[2179] Specific operation example

[2180] For example, when a store staff member is explaining a new product to a customer, the process is as follows:

[2181] 1. Step 1: When the staff member says, "This product is great. Please try it," the voice is recorded by the microphone in the smart glasses and converted into text by speech recognition software.

[2182] 2. Step 2: The server inputs the converted text, "This product is very good. Please try it.", into a natural language processing model and analyzes the conversation. The analysis results point out the lack of specificity and the use of general phrases.

[2183] 3. Step 3: Based on the analysis results, the server generates feedback such as, "It would be better to explain the product's features in more detail. For example, it would be effective to provide a specific example such as, 'This product uses the latest technology and is 30% more efficient than conventional products.'"

[2184] 4. Step 4: The generated feedback is displayed on the smart glasses display, allowing staff to receive specific advice in real time.

[2185] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2186] "Example 1"

[2187] One embodiment of the present invention provides a system that uses a natural language processing model incorporating an emotion engine. This system recognizes the user's emotions from the crew's conversation and suggests appropriate responses to the crew based on those emotions. For example, if the user uses an expression that indicates anger or frustration, the emotion engine detects this and instructs the crew to respond to the user's emotions, such as by apologizing or proposing a solution.

[2188] "Example 2"

[2189] The emotion engine also extracts patterns and phrases from the crew's conversations that reflect the user's emotions. This makes it possible to grasp changes in the user's emotions in real time and suggest appropriate responses to the crew. For example, if the user uses expressions that show joy or satisfaction, the emotion engine will detect this and instruct the crew on how to maintain the user's positive emotions, such as expressing gratitude to the crew.

[2190] "Example 3"

[2191] Furthermore, the emotion engine provides feedback to the crew member on the results of corrections to their chat and the user's emotional recognition. This allows the crew member to understand how their chat affected the user's emotions and improve their skills based on the results. For example, if the crew member's proposed solution resolves the user's dissatisfaction and the user's emotions change to joy, the crew member can confirm through this feedback that their response was appropriate.

[2192] The processing flow of each embodiment will be described below.

[2193] "Example 1"

[2194] Step 1: The crew member initiates a conversation with the user.

[2195] Step 2: The natural language processing model analyzes the crew's talk, and the emotion engine recognizes the user's emotions.

[2196] Step 3: The system suggests a response to the crew based on the user's emotions recognized by the emotion engine.

[2197] "Example 2"

[2198] Step 1: The crew member initiates a conversation with the user.

[2199] Step 2: The emotion engine extracts patterns and phrases from the crew's conversation that reflect the user's emotions.

[2200] Step 3: Based on the patterns and phrases extracted by the emotion engine, the system suggests responses to the crew.

[2201] "Example 3"

[2202] Step 1: The crew member initiates a conversation with the user.

[2203] Step 2: The natural language processing model analyzes the crew's talk, and the emotion engine recognizes the user's emotions.

[2204] Step 3: The system provides feedback to the crew member on the results of corrections to thei...

Claims

1. A natural language processing model that has learned the talk of high-productivity crew members and the talk of low-productivity crew members, A means for the natural language processing model to analyze the crew's talk and determine whether the crew's talk is good or bad; A means for the natural language processing model to correct the talk of the crew; A means for collecting crew talk data and transmitting the data to a server; A means for analyzing the emotions of a user interacting with the crew using an emotion engine capable of recognizing the emotions of the user; A means for integrating the crew's talk skill enhancements and improvements and the user's emotion recognition results to generate feedback; means for providing the generated feedback to said crew; A system including:

2. The natural language processing model extracts a specific pattern from the talk of the crew, and judges the quality of the talk of the crew based on the extracted specific pattern. The system of claim 1 .

3. A means for extracting emotions from the user's utterances and detecting negative emotions by the emotion engine; and a means for suggesting a response to the user to the crew using a natural language processing model based on a prompt sentence corresponding to the emotion detected by the emotion engine. The system of claim 1 .

Images