system

Abstract

Provide a system. 【Solution means】 Means for collecting the behavioral information of human resources in the sales operation, Means for centrally managing the collected information and storing it in an immediately accessible storage medium, Means for interpreting the response content and creating an evaluation using language data processing, Means for immediately providing a reverse feed based on the interpretation result, Means for automatically optimizing the work schedule considering the working hours and workload, Means for proposing an education plan based on individual behavioral results, Means for immediately displaying customer information and product information when human resources conduct sales activities using a visual assistance device, A system including means for analyzing the dialogue with customers through voice recognition and language analysis and providing information based on the results.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including a directive related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When there are a large number of personnel in the sales business, it is difficult to improve efficient operation and consistent performance due to the decentralized management of each business data. In addition, there are variations in the capabilities and performances of the crew, and there is a lack of appropriate feedback and training to support individual growth. As a result, business efficiency decreases, which has a negative impact on the overall operation effect.

Means for Solving the Problems

[0005] This invention provides a system for collecting and centrally managing work data of personnel engaged in sales operations in real time. This system incorporates a means for analyzing customer service content using natural language processing and providing real-time feedback. Furthermore, it achieves efficient staffing by automatically optimizing shifts based on work data, and supports efficiency and personnel growth by proposing training programs tailored to each individual's work performance.

[0006] "Sales activities" refer to a series of activities aimed at providing products or services to customers and receiving payment in return.

[0007] "Human resources" refers to employees or staff who engage in specific duties or tasks and provide value.

[0008] "Business data" refers to data that records information related to business activities, including sales, shift details, and customer service history.

[0009] "Centralized management" means consolidating multiple data sets and managing them integrally on a single platform or system.

[0010] "Real-time" refers to a state where processing and responses occur with virtually no delay, in a timeframe close to actual time.

[0011] "Natural language processing" refers to the technology used to process, understand, or generate human language using computers.

[0012] "Analysis" refers to the process of thoroughly examining data and information and interpreting its meaning.

[0013] "Feedback" refers to evaluations or opinions given regarding specific actions or results, and may include suggestions for improvement.

[0014] A "shift" refers to a work schedule for a specific time period, indicating who will perform which tasks at what time.

[0015] "Optimization" refers to the process of making adjustments and improvements to obtain the most desirable results and values under constraint conditions.

[0016] "Training program" refers to the process or curriculum of education and training designed to improve specific skills and knowledge.

Brief Description of Drawings

[0017] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [[ID=四十八]] [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Embodiment for Carrying out the Invention

[0018] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] To implement this invention, it is necessary to build a system for efficiently collecting and analyzing personnel work data in sales operations and for providing optimal feedback and personnel training. This system includes a server, terminals, and user interfaces.

[0039] In this system, each terminal first records sales data, customer service conversations, and shift information generated during the crew's work in real time. The terminals periodically send this information to a server, which stores the received data in a cloud database. This centralized management prevents data dispersion and duplication, ensuring that the latest information is always available.

[0040] Furthermore, the server uses natural language processing technology to thoroughly analyze the text data of customer interactions. This allows for the evaluation of customer service and the measurement of communication skills and service quality. The analysis results are provided as feedback from the server to each crew member's terminal, which users can then use to improve their daily work. The feedback also includes specific areas for improvement and success stories, thus promoting the growth of the crew members.

[0041] The server also automatically optimizes shifts by considering each crew member's working hours and workload. This maximizes the efficiency of staffing in store operations and helps to equalize working hours. For example, prioritizing the deployment of highly capable crew members during peak hours can improve customer satisfaction.

[0042] In addition, the server utilizes Google's AI technology to generate and propose training programs based on each individual's work performance. For example, if a crew member is analyzed as lacking knowledge of a particular product, the server will recommend an e-learning course to supplement that knowledge, which the user can then use for self-study.

[0043] In this way, by providing efficient data collection, analysis, and feedback in sales operations, as well as an optimal training platform, it is possible to build a system that improves overall operational efficiency and fosters the growth of individual crew members.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The terminal records sales data entered by crew members during sales activities, customer conversations, and shift information in real time, and periodically sends this information to the server.

[0047] Step 2:

[0048] The server stores the collected data in a cloud database. During this process, data cleansing is performed to remove incomplete or duplicate data and organize it into a unified set of information.

[0049] Step 3:

[0050] The server uses natural language processing technology to analyze text data of customer interactions. This allows for the evaluation of customer service and the measurement of communication skills and service quality.

[0051] Step 4:

[0052] Based on the analysis results, the server generates feedback including areas for improvement and success stories, and provides it to each crew member's terminal in real time. Users then use this feedback to improve their work processes.

[0053] Step 5:

[0054] The server uses IBM's optimization engine to automatically optimize shifts based on crew working hours and workload. This enables efficient staffing.

[0055] Step 6:

[0056] The server analyzes each crew member's work performance data and automatically generates the optimal training program based on that data. Leveraging Google's AI technology, the server proposes specific training and learning plans to the user.

[0057] Step 7:

[0058] Users can access training programs and feedback provided through their devices to aid in self-study and skill development. Each crew member uses this information to improve their performance and contribute to their own growth.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] In sales operations, improving overall operational efficiency while promoting efficient performance enhancement and growth among workers requires the rapid and accurate collection of individual worker activity data and the provision of feedback based on the analysis results. However, conventional methods have made it difficult to achieve this effectively. The main reasons for this include the difficulty of real-time data management, insufficient quality of feedback, and the manual adjustment of working hours and workload.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes means for acquiring worker activity data in sales operations, means for centrally managing the acquired data and storing it in an information repository for immediate use, and means for analyzing the content of interactions and creating evaluations using natural language processing technology. This enables real-time data management and highly effective feedback based on analysis results, thereby promoting the efficiency and growth of workers.

[0064] "Sales operations" refers to all activities aimed at providing goods and services to customers, and primarily includes commercial transactions and customer service conducted within a store.

[0065] "Workers" refers to individual staff or employees engaged in sales operations, who, as part of their duties, interact with customers and engage in sales activities.

[0066] "Activity data" refers to all information related to an employee's work performance, including sales information, customer service details, and shift data.

[0067] "Centralized management" refers to a management method that involves consolidating dispersed data into a single information repository for easy access and use.

[0068] An "information storage facility" refers to a database or data storage system used to centrally store collected data.

[0069] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and process the language that humans use in everyday life, and is used for analyzing text and understanding intent.

[0070] "Content of interaction" refers to the content of communication between the worker and the customer, and includes conversations and question-and-answer sessions in customer service.

[0071] "Evaluation" refers to judgments and ratings of workers and their activities obtained based on the analysis results.

[0072] "Immediately available" means that data is ready to be processed and used as soon as it is collected, enabling real-time access.

[0073] To implement this invention, it is necessary to construct a system that efficiently acquires and manages activity data of workers engaged in sales operations, and provides analysis and feedback based on that data. The system includes an internet-connected terminal, a server for data processing, and a user interface for providing information.

[0074] The terminal records activity data in real time, such as sales data, customer service details, and shift information, as employees perform their duties in the store. This utilizes POS systems and voice recognition software. The terminal periodically transmits the collected information to a server via a secure protocol.

[0075] The server centralizes and manages the received data and stores it in a cloud-based information repository. Using advanced natural language processing technology, the server analyzes the text data of customer service interactions and evaluates the quality of customer service. Common natural language APIs are used for this processing. Based on the analysis results, the server generates immediate feedback and provides it to the worker via their terminal. The feedback includes specific areas for improvement and success stories, allowing workers to use it to improve the quality of their work.

[0076] Users receive feedback from the server, which helps them improve their daily work. This process allows workers to continuously improve their work performance.

[0077] As a concrete example, if the server analyzes the content of customer service conversations recorded on a terminal and finds that the response to the customer's request was insufficient, the server will provide feedback with suggestions for improving the conversation. Based on these suggestions, the user can learn how to improve their customer service.

[0078] A useful example of a prompt would be: "We want to analyze customer service conversation data and evaluate customer service. Please tell us how to provide feedback on specific areas for improvement and successful cases."

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] The terminal acquires sales data, customer service details, and shift information in real time as input while the worker is active. Specifically, it captures transaction data from the POS system and transcribes conversations into text using speech recognition software. As a result, various business data is output in digital format.

[0082] Step 2:

[0083] The terminal encrypts the acquired data and sends it to the server as input using a secure communication protocol. Specifically, this involves data transmission using SSL / TLS. The data is then securely delivered to the server as output.

[0084] Step 3:

[0085] The server stores the received data as input in a cloud-based information repository. During this process, data duplication checks and format conversions are performed, and a centralized database is constructed as output.

[0086] Step 4:

[0087] The server extracts text data of customer service interactions from a cloud-based database and analyzes it using natural language processing technology. Specifically, it performs keyword extraction and sentiment analysis to generate customer service evaluations. The results of the analysis are output as evaluation scores and recommended areas for improvement.

[0088] Step 5:

[0089] The server generates feedback as input based on the analysis results. Specifically, it creates reports that clearly indicate areas for improvement and guidelines that incorporate successful case studies. This is then sent as output to the terminal, notifying the worker.

[0090] Step 6:

[0091] Users receive feedback via their devices as input and utilize it to improve their work. Specifically, they use this information to review their daily customer interactions, reflect it in self-evaluations and team meetings, and use it as output to improve their skills.

[0092] (Application Example 1)

[0093] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0094] In the current sales environment, it is difficult to efficiently utilize information on the behavior of human resources and the content of customer interactions to provide appropriate feedback on the spot and improve the quality of sales. As a result, progress in operational efficiency and improvement of customer service may not be made, potentially leading to a decline in competitiveness. Furthermore, providing effective training plans requires a great deal of time and effort, posing challenges to the growth of individual human resources.

[0095] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0096] In this invention, the server includes means for collecting behavioral information of human resources in sales operations, means for instantly displaying customer and product information when human resources are performing sales activities using a visual assistance device, and means for analyzing customer interactions through speech recognition and language analysis and providing information based on the results. This makes it possible to immediately provide information tailored to the customer and improve the quality of sales. Furthermore, it is possible to effectively support the growth of human resources through the proposal of training plans based on individual work results.

[0097] "Sales activities" refer to a series of activities or processes for providing goods or services to customers.

[0098] "Human resources" refers to the personnel necessary for an organization or company to utilize people's work abilities and experience.

[0099] "Behavioral information" refers to data and records concerning the actions and activities of an individual or group.

[0100] "Visual assistance devices" refer to technological devices that present information to users visually, and smart glasses are an example of such devices.

[0101] "Speech recognition" is a technical process that acquires speech data and converts it into text or commands.

[0102] "Linguistic analysis" is a technique for extracting and interpreting information contained in natural language.

[0103] An "educational plan" is an organized set of learning content and methods aimed at acquiring specific knowledge and skills.

[0104] A "database" is an information system that organizes and manages a collection of related data.

[0105] In this embodiment of the invention, a server, a terminal, and a user cooperate to form a system. The server has the function of collecting behavioral information of human resources in sales operations and storing it in a centrally managed storage medium. This information is accessible in real time, and the server processes the information immediately and generates evaluations.

[0106] The terminal functions as a visual aid, displaying customer and product information within the user's field of vision during sales activities. This allows the user to provide appropriate product suggestions and services based on the content of their conversations with customers. Using speech recognition technology to transcribe customer conversations into text and language analysis technology to interpret the content, the server provides accurate feedback to the user.

[0107] As a concrete example, if a user is assigned to a bookstore and a customer is looking for a novel of a specific genre, the terminal analyzes the customer's speech and displays relevant book information in real time. This allows the customer to quickly find the product they are looking for.

[0108] Furthermore, this system also includes a function to propose educational plans based on individual performance outcomes. The server analyzes each user's work data and automatically generates educational plans to improve specific skills and knowledge as needed. Users can then utilize these plans for self-improvement.

[0109] Example prompt: "A customer came into the bookstore and said they like mystery novels. Please select a book to recommend and provide some customer service tips."

[0110] This system is expected to create an environment that supports increased efficiency in sales operations and the growth of human resources.

[0111] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0112] Step 1:

[0113] The server receives information on the behavior of human resources and the content of customer interactions as audio data from the terminal. Using this audio data as input, the server uses the Google Speech-to-Text API to convert it from audio to text data. The resulting text data is then output.

[0114] Step 2:

[0115] The server uses the Google Cloud Natural Language API to perform language analysis on the text data obtained in Step 1. Here, customer needs are extracted from the text data, and related information is analyzed. The analysis results output information about customer needs.

[0116] Step 3:

[0117] The server compares the analysis results from step 2 with the product information stored on the storage medium and generates a list of products that meet the customer's needs. This product list is output and sent to the terminal, where it is displayed on the user's visual terminal.

[0118] Step 4:

[0119] Based on product information displayed on a visual terminal, users suggest recommended products to customers. Some of the user's actions are recorded again as behavioral data and used for future feedback.

[0120] Step 5:

[0121] The server uses a generative AI model, taking user work data and past feedback as input, to automatically generate training plans tailored to the user's work performance. This training plan is then output and presented to the user via their terminal.

[0122] By efficiently processing data at each step and providing feedback to users, the overall quality of sales operations can be improved.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] This invention is an efficient human resource management system for sales operations that incorporates an emotion engine. It promotes operational efficiency and crew growth through the collection and analysis of operational data, appropriate feedback, and the proposal of training programs. The emotion engine allows for the recognition of users' emotional states and their reflection in their work.

[0125] The terminal records sales information, customer conversations, and work shift data entered by crew members during sales activities in real time. This data is sent to a server and centrally managed. The server analyzes the collected data using natural language processing technology to evaluate the quality of customer service and communication skills.

[0126] Furthermore, the server is equipped with an emotion engine that can detect in real time the emotions and stress levels that crew members exhibit while serving customers. For example, if the emotion engine detects that a crew member's stress levels are rising based on their facial expressions or tone of voice, the analysis results are reflected in the server, and appropriate feedback is provided.

[0127] This feedback is sent to each crew member's terminal, allowing users to adjust their customer service style and approach according to their own emotional state. For example, if the emotion engine detects tension during a crew member's interaction with a customer, the server provides feedback such as suggestions for relaxation techniques or the next point of focus.

[0128] Furthermore, the server automatically generates training programs tailored to individual needs based on each crew member's performance evaluation and emotional data. These training programs are delivered in the form of e-learning courses and in-person sessions, serving as a guide for users' growth.

[0129] In this way, this system, which incorporates an emotional engine, understands the emotional state of the crew and supports work efficiency and individual growth through appropriate feedback and training.

[0130] The following describes the processing flow.

[0131] Step 1:

[0132] The terminal records sales information and customer conversations entered by the crew during sales activities in real time, and transmits this information to the server.

[0133] Step 2:

[0134] The server receives data sent from terminals and centrally manages it in a cloud database. It also performs data cleansing and organizes any data that is corrupted.

[0135] Step 3:

[0136] The server uses natural language processing technology to analyze customer service interactions and evaluate the quality of service and communication skills. The analysis results serve as the basis for feedback.

[0137] Step 4:

[0138] The server has an emotion engine built in that detects the emotional state of the crew in real time from their tone of voice and facial expressions. It identifies when they are emotionally agitated or under high stress.

[0139] Step 5:

[0140] The server generates immediate feedback based on the analyzed sentiment data and customer service evaluation, and sends it to the user's device. This allows the user to understand areas for improvement in their customer service style.

[0141] Step 6:

[0142] Users who see feedback that includes data from the emotion engine can use that feedback to adjust their customer service methods. For example, they might be mindful of relaxation techniques or points to keep in mind for future interactions.

[0143] Step 7:

[0144] The server automatically generates training programs to promote growth based on each crew member's performance evaluation and emotional data. The generated programs are then provided to users via their terminals.

[0145] Step 8:

[0146] Users aim to improve their skills and obtain certifications by following training programs provided on their devices. Through this process, users can continuously improve themselves.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] To improve the operational efficiency of personnel in sales operations, it is necessary not only to collect operational data, but also to accurately grasp the actual performance and emotional state of individual employees, provide immediate feedback, and offer appropriate training. However, conventional systems do not integrate these elements, which has led to challenges such as decreased operational efficiency and stifled opportunities for individual growth.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes means for collecting work information related to sales operations, means for centrally controlling the collected information and storing it in an information storage facility that can be accessed immediately, means for analyzing customer interaction content and emotional state and generating evaluations using natural language processing and sentiment analysis engines, means for providing responses in real time based on the analysis results and sentiment analysis results, means for automatically optimizing work shifts considering working hours and workload, and means for providing educational programs based on each individual's work evaluation and emotional state. This makes it possible to provide feedback and educational programs from both work data and emotional perspectives, thereby simultaneously achieving improved work efficiency and employee growth.

[0152] "Sales operations" refer to a series of activities related to the provision and transaction of goods, and include operations that involve direct interaction with customers.

[0153] "Work information" refers to data about the specific activities performed by employees in a business process and the results thereof.

[0154] "Centralized control" refers to a state where collected data is centrally managed through a central system or platform, and access and processing are integrated.

[0155] An "information storage facility" is a storage device or database used to store various types of data and make them readily accessible as needed.

[0156] "Natural language processing" refers to the technology of processing human language using computers and interpreting its meaning and intent.

[0157] An "emotion analysis engine" is a system that recognizes a person's emotional state from their voice, facial expressions, and other information, and performs analysis based on that information.

[0158] "Generating an evaluation" means quantifying or judging the performance of an individual or behavior according to specific criteria, based on collected data.

[0159] "Providing real-time responses" means providing feedback and suggestions for improvement almost immediately based on collected and analyzed data.

[0160] "Automatically optimizing work shifts based on working hours and workload" is a process that automatically readjusts schedules to efficiently adjust employee work effort and time management, thereby improving labor efficiency.

[0161] An "educational program" is a general term for training and guidance activities aimed at improving employees' skills and knowledge.

[0162] This invention is a system that promotes the work efficiency and individual growth of employees in sales operations. Specifically, it is achieved by collecting work information from the sales floor in real time and using that information for evaluation and feedback.

[0163] The terminals collect sales information and customer conversation details using voice input and touch controls while each employee is performing sales duties. This data is immediately transmitted to the server.

[0164] The server stores the collected data in an information repository and controls it centrally. Furthermore, the server is equipped with natural language processing software and an emotion analysis engine to analyze the transmitted data. This analysis generates evaluations of customer service and the emotional states of employees.

[0165] The server has the ability to provide real-time feedback to each employee based on the analysis results. This feedback is delivered immediately to the terminal and includes specific advice, such as, "Your voice is trembling, so try speaking more slowly to calm yourself down."

[0166] Furthermore, the server automatically proposes optimal shifts, taking into account employee working hours and workload. Based on work evaluations and sentiment analysis, it also suggests training programs that will maximize the benefits for each employee. These programs are delivered in e-learning or in-person session formats.

[0167] For example, when a cafe employee receives an order for "two cafe lattes" at the counter, real-time feedback such as "It's busy today, so please try to work efficiently" is provided. Another example of a prompt message is: "Generate feedback to improve customer service skills in the cafe. Situation: Employee is stressed. Needed feedback: Relaxation techniques and the next steps in customer service."

[0168] Thus, the present invention is a system that simultaneously achieves work efficiency and personal growth by integrating and analyzing work data and emotional states, and providing corresponding feedback and educational programs.

[0169] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0170] Step 1:

[0171] The terminal acquires sales information and customer interaction transcripts entered by employees during sales activities. This information is collected through touch input and voice recognition. Input data includes specific product names, quantities, and transcripts of customer conversations. The collected information is transmitted to the server in real time.

[0172] Step 2:

[0173] The server receives data sent from the terminal and stores it in the information repository. Next, natural language processing technology is used to convert the content of customer interactions into structured data. This data processing makes it possible to evaluate customer interactions in detail. For example, keywords from customer service are extracted and the flow of the conversation is logically structured, and an evaluation output is generated.

[0174] Step 3:

[0175] The server uses an emotion analysis engine to analyze the tone of voice and language patterns of employees. Input data includes voice and text information, and the analysis results in the output of the employee's emotional state, such as stress level and tension level. These results are then used in the subsequent feedback generation process.

[0176] Step 4:

[0177] The server generates real-time feedback based on natural language processing and sentiment analysis results. This feedback includes specific improvement advice and encouraging messages regarding customer service methods. The generated feedback is sent to the terminal, and the user receives it and uses it as information to quickly correct their response.

[0178] Step 5:

[0179] The server continuously monitors each employee's working hours and workload, and automatically generates optimized work shifts. It uses operational performance and workload data as input, and provides an efficient schedule plan as output. This reduces the burden on employees and improves operational efficiency.

[0180] Step 6:

[0181] Finally, the server combines each employee's performance evaluation and emotional data to propose a personalized training program. Past evaluation history and emotional states are used as input data, and a personalized training plan is generated as output. This plan is provided to the user as e-learning materials or in-person sessions to support skill improvement.

[0182] (Application Example 2)

[0183] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0184] Traditional human resource management systems often fail to adequately consider the emotional state of sales staff and their interactions with customers, making it difficult to optimize operational efficiency and individual growth. Therefore, there is a need for technologies that can improve the quality of customer service, reduce staff stress, and provide effective training.

[0185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0186] In this invention, the server includes means for collecting the status of members involved in sales operations and sales data; means for centrally managing the collected information and storing it in a storage device that can be accessed in real time; means for analyzing customer service content and generating evaluations using natural language processing technology; means for providing immediate feedback to members based on the analysis results; means for analyzing the status of customers and members using emotional data from devices worn by members; means for automatically optimizing work schedules considering working hours and workload; and means for proposing training plans based on individual work evaluations. This enables flexible work operations that take into account the emotional state of the crew, improved quality of customer service, and effective training based on individual needs.

[0187] "Members involved in sales operations" refers to individuals who are responsible for providing goods and services at stores or sales locations.

[0188] "Sales data" refers to data that includes information related to sales operations, such as product sales, customer information, and purchase history.

[0189] "Centralized management" refers to maintaining a state where information and data are centrally managed and easily accessible as needed.

[0190] A "real-time accessible storage device" refers to a data storage system where information is updated instantly and can be accessed immediately as needed.

[0191] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0192] "Analyzing customer service content and generating evaluations" refers to the process of thoroughly investigating the content of customer interactions and creating evaluations based on the results.

[0193] "Using emotional data to analyze the state of customers and team members" refers to evaluating the mental state of customers and crew members using emotional data obtained from facial expressions, voice, and other sources.

[0194] "Automatically optimizing work schedules" refers to the process of automatically creating the optimal work schedule by taking into account working hours and workload.

[0195] "Proposing a training plan based on individual performance evaluations" refers to the act of proposing a training program to support the growth of each team member based on an evaluation of their work performance.

[0196] In the system realizing this invention, the server utilizes various devices and software to manage and analyze information obtained from sales activities in which members participate. Specifically, the server collects sales data and sentiment data recorded by members during their work and stores them centrally in a storage device. This enables real-time data access.

[0197] The terminals refer to smart glasses and smartphones worn by crew members, which acquire video and audio from these devices and transmit them to a server. The server analyzes this data using natural language processing technology to evaluate the quality of customer service. The analysis results are immediately fed back to the team members, who then provide specific suggestions for improving customer service methods and reducing stress.

[0198] The server utilizes an emotion engine to analyze the emotional states of crew members and customers in real time. This allows it to provide useful information for improving the quality of customer service. For example, if a crew member is feeling stressed, the server will suggest relaxation techniques. The server also has the ability to estimate workload and automatically optimize work schedules based on this.

[0199] For example, if this system were used in a department store's physical location, schedules would be adjusted to allow crew members to take appropriate breaks even during busy periods, and a phase would be presented where additional information about expensive items is provided when a customer shows interest in them.

[0200] An example of a prompt using a generative AI model would be: "Please explain how you alleviate customer anxiety during today's customer service. Also, if a crew member is feeling stressed during work, how would you help them relax?"

[0201] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0202] Step 1:

[0203] The server receives sales data and audio recordings of customer conversations transmitted from the terminal. The input data consists of information collected during the sales process, which the server centrally stores in storage, enabling real-time access.

[0204] Step 2:

[0205] The server uses natural language processing technology to analyze stored conversation data. The input data is conversational information in audio format, which the server converts to text and performs analysis to evaluate the quality of customer service. As output, it generates evaluation data regarding the quality of customer service. Specifically, it analyzes keywords and tone in the conversation and quantifies customer responses.

[0206] Step 3:

[0207] The server sends real-time feedback to the crew based on the evaluation results. The input here is the evaluated customer service, and the server uses this to generate specific areas for improvement and customer service advice, which is then output to the terminal. This feedback includes encouragement and suggestions for improvement.

[0208] Step 4:

[0209] The terminal transmits video data obtained from smart glasses worn by the crew to the emotion engine. The video data is provided as input, and the emotion engine analyzes the facial expressions of the crew and customers to determine their emotional state. The output is data indicating the emotional state. This enables real-time emotional assessment.

[0210] Step 5:

[0211] After receiving emotional data, the server provides the crew with specific feedback and action plans tailored to their emotional state. The input is data indicating emotional state, and the server generates suggestions such as relaxation techniques and adjustments to customer service style, which are then output to the terminal. Specifically, this includes advice on stress level adjustments and customer service tone.

[0212] Step 6:

[0213] The server aggregates individual crew members' performance evaluations and emotional data to optimize work schedules. It uses member work data and emotional data as input to calculate appropriate working hours and break times. The output provides an optimized work schedule, specifically adjusting it to account for workload and peak times.

[0214] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0215] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0216] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0217] [Second Embodiment]

[0218] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0219] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0220] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0221] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0222] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0223] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0224] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0225] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0226] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0227] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0228] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0229] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0230] To implement this invention, it is necessary to build a system for efficiently collecting and analyzing personnel work data in sales operations and for providing optimal feedback and personnel training. This system includes a server, terminals, and user interfaces.

[0231] In this system, each terminal first records sales data, customer service conversations, and shift information generated during the crew's work in real time. The terminals periodically send this information to a server, which stores the received data in a cloud database. This centralized management prevents data dispersion and duplication, ensuring that the latest information is always available.

[0232] Furthermore, the server uses natural language processing technology to thoroughly analyze the text data of customer interactions. This allows for the evaluation of customer service and the measurement of communication skills and service quality. The analysis results are provided as feedback from the server to each crew member's terminal, which users can then use to improve their daily work. The feedback also includes specific areas for improvement and success stories, thus promoting the growth of the crew members.

[0233] The server also automatically optimizes shifts by considering each crew member's working hours and workload. This maximizes the efficiency of staffing in store operations and helps to equalize working hours. For example, prioritizing the deployment of highly capable crew members during peak hours can improve customer satisfaction.

[0234] In addition, the server utilizes Google's AI technology to generate and propose training programs based on each individual's work performance. For example, if a crew member is analyzed as lacking knowledge of a particular product, the server will recommend an e-learning course to supplement that knowledge, which the user can then use for self-study.

[0235] In this way, by providing efficient data collection, analysis, and feedback in sales operations, as well as an optimal training platform, it is possible to build a system that improves overall operational efficiency and fosters the growth of individual crew members.

[0236] The following describes the processing flow.

[0237] Step 1:

[0238] The terminal records sales data entered by crew members during sales activities, customer conversations, and shift information in real time, and periodically sends this information to the server.

[0239] Step 2:

[0240] The server stores the collected data in a cloud database. During this process, data cleansing is performed to remove incomplete or duplicate data and organize it into a unified set of information.

[0241] Step 3:

[0242] The server uses natural language processing technology to analyze text data of customer interactions. This allows for the evaluation of customer service and the measurement of communication skills and service quality.

[0243] Step 4:

[0244] Based on the analysis results, the server generates feedback including areas for improvement and success stories, and provides it to each crew member's terminal in real time. Users then use this feedback to improve their work processes.

[0245] Step 5:

[0246] The server uses IBM's optimization engine to automatically optimize shifts based on crew working hours and workload. This enables efficient staffing.

[0247] Step 6:

[0248] The server analyzes each crew member's work performance data and automatically generates the optimal training program based on that data. Leveraging Google's AI technology, the server proposes specific training and learning plans to the user.

[0249] Step 7:

[0250] Users can access training programs and feedback provided through their devices to aid in self-study and skill development. Each crew member uses this information to improve their performance and contribute to their own growth.

[0251] (Example 1)

[0252] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0253] In sales operations, improving overall operational efficiency while promoting efficient performance enhancement and growth among workers requires the rapid and accurate collection of individual worker activity data and the provision of feedback based on the analysis results. However, conventional methods have made it difficult to achieve this effectively. The main reasons for this include the difficulty of real-time data management, insufficient quality of feedback, and the manual adjustment of working hours and workload.

[0254] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0255] In this invention, the server includes means for acquiring worker activity data in sales operations, means for centrally managing the acquired data and storing it in an information repository for immediate use, and means for analyzing the content of interactions and creating evaluations using natural language processing technology. This enables real-time data management and highly effective feedback based on analysis results, thereby promoting the efficiency and growth of workers.

[0256] "Sales operations" refers to all activities aimed at providing goods and services to customers, and primarily includes commercial transactions and customer service conducted within a store.

[0257] "Workers" refers to individual staff or employees engaged in sales operations, who, as part of their duties, interact with customers and engage in sales activities.

[0258] "Activity data" refers to all information related to an employee's work performance, including sales information, customer service details, and shift data.

[0259] "Centralized management" refers to a management method that involves consolidating dispersed data into a single information repository for easy access and use.

[0260] An "information storage facility" refers to a database or data storage system used to centrally store collected data.

[0261] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and process the language that humans use in everyday life, and is used for analyzing text and understanding intent.

[0262] "Content of interaction" refers to the content of communication between the worker and the customer, and includes conversations and question-and-answer sessions in customer service.

[0263] "Evaluation" refers to judgments and ratings of workers and their activities obtained based on the analysis results.

[0264] "Immediately available" means that data is ready to be processed and used as soon as it is collected, enabling real-time access.

[0265] To implement this invention, it is necessary to construct a system that efficiently acquires and manages activity data of workers engaged in sales operations, and provides analysis and feedback based on that data. The system includes an internet-connected terminal, a server for data processing, and a user interface for providing information.

[0266] The terminal records activity data in real time, such as sales data, customer service details, and shift information, as employees perform their duties in the store. This utilizes POS systems and voice recognition software. The terminal periodically transmits the collected information to a server via a secure protocol.

[0267] The server centralizes and manages the received data and stores it in a cloud-based information repository. Using advanced natural language processing technology, the server analyzes the text data of customer service interactions and evaluates the quality of customer service. Common natural language APIs are used for this processing. Based on the analysis results, the server generates immediate feedback and provides it to the worker via their terminal. The feedback includes specific areas for improvement and success stories, allowing workers to use it to improve the quality of their work.

[0268] Users receive feedback from the server, which helps them improve their daily work. This process allows workers to continuously improve their work performance.

[0269] As a concrete example, if the server analyzes the content of customer service conversations recorded on a terminal and finds that the response to the customer's request was insufficient, the server will provide feedback with suggestions for improving the conversation. Based on these suggestions, the user can learn how to improve their customer service.

[0270] A useful example of a prompt would be: "We want to analyze customer service conversation data and evaluate customer service. Please tell us how to provide feedback on specific areas for improvement and successful cases."

[0271] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0272] Step 1:

[0273] The terminal acquires sales data, customer service details, and shift information in real time as input while the worker is active. Specifically, it captures transaction data from the POS system and transcribes conversations into text using speech recognition software. As a result, various business data is output in digital format.

[0274] Step 2:

[0275] The terminal encrypts the acquired data and transmits it to the server as input using a secure communication protocol. Specific operations here include data transmission using SSL / TLS. As output, the data reaches the server securely.

[0276] Step 3:

[0277] The server stores the received data as input in a cloud-based information repository. At this time, duplicate data checking and format conversion are performed, and a unified database is constructed as output.

[0278] Step 4:

[0279] The server extracts text data on customer service content from the database on the cloud as input and performs analysis using natural language processing technology. Specifically, keyword extraction and sentiment analysis are performed to generate an evaluation of customer service. The results of the analysis are output as an evaluation score and recommended improvement points.

[0280] Step 5:

[0281] The server generates feedback as input based on the analysis results. Specifically, it creates a report highlighting improvement points and guidelines incorporating successful cases. This is output and transmitted to the terminal to notify the operator.

[0282] Step 6:

[0283] The user receives the feedback received through the terminal as input and utilizes it for business improvement. As a specific operation, based on this information, the daily customer service is reviewed, reflected in self-evaluation and team meetings, and the output is aimed at improving skills.

[0284] (Application Example 1)

[0285] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0286] In the current sales operation environment, it is difficult to efficiently utilize the action information of human resources and the content of conversations with customers, and to achieve appropriate feedback and improvement of sales quality on the spot. As a result, the efficiency of operations and customer response cannot be improved, which may lead to a decline in competitiveness. Furthermore, providing an effective education program requires a lot of time and effort, and there are challenges in the growth of individual human resources.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0288] In this invention, the server includes means for collecting the action information of human resources in sales operations, means for immediately displaying customer information and product information when human resources conduct sales activities using a visual assistance device, and means for analyzing conversations with customers through voice recognition and language analysis and providing information based on the results. As a result, it becomes possible to immediately provide information suitable for customers and improve the quality of sales. Furthermore, the growth of human resources can be effectively supported through the proposal of an education program based on individual business results.

[0289] "Sales operations" refers to a series of activities or processes for providing goods and services to customers.

[0290] "Human resources" refers to the personnel necessary for an organization or company to utilize the labor ability and experience of people.

[0291] "Action information" refers to data and records related to the actions and activities of an individual or group.

[0292] "Visual assistance device" refers to a technical device for visually presenting information to a user, and includes smart glasses as an example.

[0293] "Speech recognition" is a technical process that acquires speech data and converts it into text or commands.

[0294] "Linguistic analysis" is a technique for extracting and interpreting information contained in natural language.

[0295] An "educational plan" is an organized set of learning content and methods aimed at acquiring specific knowledge and skills.

[0296] A "database" is an information system that organizes and manages a collection of related data.

[0297] In this embodiment of the invention, a server, a terminal, and a user cooperate to form a system. The server has the function of collecting behavioral information of human resources in sales operations and storing it in a centrally managed storage medium. This information is accessible in real time, and the server processes the information immediately and generates evaluations.

[0298] The terminal functions as a visual aid, displaying customer and product information within the user's field of vision during sales activities. This allows the user to provide appropriate product suggestions and services based on the content of their conversations with customers. Using speech recognition technology to transcribe customer conversations into text and language analysis technology to interpret the content, the server provides accurate feedback to the user.

[0299] As a concrete example, if a user is assigned to a bookstore and a customer is looking for a novel of a specific genre, the terminal analyzes the customer's speech and displays relevant book information in real time. This allows the customer to quickly find the product they are looking for.

[0300] Furthermore, this system also includes a function to propose educational plans based on individual performance outcomes. The server analyzes each user's work data and automatically generates educational plans to improve specific skills and knowledge as needed. Users can then utilize these plans for self-improvement.

[0301] Example of prompt text: "A customer who came to the bookstore said they like mystery novels. Please select recommended books and present the key points of customer service."

[0302] It is expected that this system will create an environment that supports the efficiency of sales operations and the growth of human resources.

[0303] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0304] Step 1:

[0305] The server receives, as voice data, the action information of human resources obtained from the terminal and the content of the conversation between the customer. Using this voice data as input, the Google Speech-to-Text API is utilized to perform a process of converting the voice into text data. The text data generated as a result is output.

[0306] Step 2:

[0307] The server performs language analysis on the text data obtained in Step 1 using the Google Cloud Natural Language API. Here, the needs of the customer are extracted from the text data, and related information is analyzed. Information regarding the needs of the customer is output as the analysis result.

[0308] Step 3:

[0309] The server collates the analysis result of Step 2 with the product information stored in the storage medium, and generates a list of products that match the needs of the customer. By outputting this product list and transmitting it to the terminal, an operation of displaying it on the user's visual terminal is performed.

[0310] Step 4:

[0311] Based on product information displayed on a visual terminal, users suggest recommended products to customers. Some of the user's actions are recorded again as behavioral data and used for future feedback.

[0312] Step 5:

[0313] The server uses a generative AI model, taking user work data and past feedback as input, to automatically generate training plans tailored to the user's work performance. This training plan is then output and presented to the user via their terminal.

[0314] By efficiently processing data at each step and providing feedback to users, the overall quality of sales operations can be improved.

[0315] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0316] This invention is an efficient human resource management system for sales operations that incorporates an emotion engine. It promotes operational efficiency and crew growth through the collection and analysis of operational data, appropriate feedback, and the proposal of training programs. The emotion engine allows for the recognition of users' emotional states and their reflection in their work.

[0317] The terminal records sales information, customer conversations, and work shift data entered by crew members during sales activities in real time. This data is sent to a server and centrally managed. The server analyzes the collected data using natural language processing technology to evaluate the quality of customer service and communication skills.

[0318] Furthermore, the server is equipped with an emotion engine that can detect in real time the emotions and stress levels that crew members exhibit while serving customers. For example, if the emotion engine detects that a crew member's stress levels are rising based on their facial expressions or tone of voice, the analysis results are reflected in the server, and appropriate feedback is provided.

[0319] This feedback is sent to each crew member's terminal, allowing users to adjust their customer service style and approach according to their own emotional state. For example, if the emotion engine detects tension during a crew member's interaction with a customer, the server provides feedback such as suggestions for relaxation techniques or the next point of focus.

[0320] Furthermore, the server automatically generates training programs tailored to individual needs based on each crew member's performance evaluation and emotional data. These training programs are delivered in the form of e-learning courses and in-person sessions, serving as a guide for users' growth.

[0321] In this way, this system, which incorporates an emotional engine, understands the emotional state of the crew and supports work efficiency and individual growth through appropriate feedback and training.

[0322] The following describes the processing flow.

[0323] Step 1:

[0324] The terminal records sales information and customer conversations entered by the crew during sales activities in real time, and transmits this information to the server.

[0325] Step 2:

[0326] The server receives data sent from terminals and centrally manages it in a cloud database. It also performs data cleansing and organizes any data that is corrupted.

[0327] Step 3:

[0328] The server uses natural language processing technology to analyze customer service interactions and evaluate the quality of service and communication skills. The analysis results serve as the basis for feedback.

[0329] Step 4:

[0330] The server has an emotion engine built in that detects the emotional state of the crew in real time from their tone of voice and facial expressions. It identifies when they are emotionally agitated or under high stress.

[0331] Step 5:

[0332] The server generates immediate feedback based on the analyzed sentiment data and customer service evaluation, and sends it to the user's device. This allows the user to understand areas for improvement in their customer service style.

[0333] Step 6:

[0334] Users who see feedback that includes data from the emotion engine can use that feedback to adjust their customer service methods. For example, they might be mindful of relaxation techniques or points to keep in mind for future interactions.

[0335] Step 7:

[0336] The server automatically generates training programs to promote growth based on each crew member's performance evaluation and emotional data. The generated programs are then provided to users via their terminals.

[0337] Step 8:

[0338] Users aim to improve their skills and obtain certifications by following training programs provided on their devices. Through this process, users can continuously improve themselves.

[0339] (Example 2)

[0340] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0341] To improve the operational efficiency of personnel in sales operations, it is necessary not only to collect operational data, but also to accurately grasp the actual performance and emotional state of individual employees, provide immediate feedback, and offer appropriate training. However, conventional systems do not integrate these elements, which has led to challenges such as decreased operational efficiency and stifled opportunities for individual growth.

[0342] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0343] In this invention, the server includes means for collecting work information related to sales operations, means for centrally controlling the collected information and storing it in an information storage facility that can be accessed immediately, means for analyzing customer interaction content and emotional state and generating evaluations using natural language processing and sentiment analysis engines, means for providing responses in real time based on the analysis results and sentiment analysis results, means for automatically optimizing work shifts considering working hours and workload, and means for providing educational programs based on each individual's work evaluation and emotional state. This makes it possible to provide feedback and educational programs from both work data and emotional perspectives, thereby simultaneously achieving improved work efficiency and employee growth.

[0344] "Sales operations" refer to a series of activities related to the provision and transaction of goods, and include operations that involve direct interaction with customers.

[0345] "Work information" refers to data about the specific activities performed by employees in a business process and the results thereof.

[0346] "Centralized control" refers to a state where collected data is centrally managed through a central system or platform, and access and processing are integrated.

[0347] An "information storage facility" is a storage device or database used to store various types of data and make them readily accessible as needed.

[0348] "Natural language processing" refers to the technology of processing human language using computers and interpreting its meaning and intent.

[0349] An "emotion analysis engine" is a system that recognizes a person's emotional state from their voice, facial expressions, and other information, and performs analysis based on that information.

[0350] "Generating an evaluation" means quantifying or judging the performance of an individual or behavior according to specific criteria, based on collected data.

[0351] "Providing real-time responses" means providing feedback and suggestions for improvement almost immediately based on collected and analyzed data.

[0352] "Automatically optimizing work shifts based on working hours and workload" is a process that automatically readjusts schedules to efficiently adjust employee work effort and time management, thereby improving labor efficiency.

[0353] An "educational program" is a general term for training and guidance activities aimed at improving employees' skills and knowledge.

[0354] This invention is a system that promotes the work efficiency and individual growth of employees in sales operations. Specifically, it is achieved by collecting work information from the sales floor in real time and using that information for evaluation and feedback.

[0355] The terminals collect sales information and customer conversation details using voice input and touch controls while each employee is performing sales duties. This data is immediately transmitted to the server.

[0356] The server stores the collected data in an information repository and controls it centrally. Furthermore, the server is equipped with natural language processing software and an emotion analysis engine to analyze the transmitted data. This analysis generates evaluations of customer service and the emotional states of employees.

[0357] The server has the ability to provide real-time feedback to each employee based on the analysis results. This feedback is delivered immediately to the terminal and includes specific advice, such as, "Your voice is trembling, so try speaking more slowly to calm yourself down."

[0358] Furthermore, the server automatically proposes optimal shifts, taking into account employee working hours and workload. Based on work evaluations and sentiment analysis, it also suggests training programs that will maximize the benefits for each employee. These programs are delivered in e-learning or in-person session formats.

[0359] For example, when a cafe employee receives an order for "two cafe lattes" at the counter, real-time feedback such as "It's busy today, so please try to work efficiently" is provided. Another example of a prompt message is: "Generate feedback to improve customer service skills in the cafe. Situation: Employee is stressed. Needed feedback: Relaxation techniques and the next steps in customer service."

[0360] Thus, the present invention is a system that simultaneously achieves work efficiency and personal growth by integrating and analyzing work data and emotional states, and providing corresponding feedback and educational programs.

[0361] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0362] Step 1:

[0363] The terminal acquires sales information and customer interaction transcripts entered by employees during sales activities. This information is collected through touch input and voice recognition. Input data includes specific product names, quantities, and transcripts of customer conversations. The collected information is transmitted to the server in real time.

[0364] Step 2:

[0365] The server receives data sent from the terminal and stores it in the information repository. Next, natural language processing technology is used to convert the content of customer interactions into structured data. This data processing makes it possible to evaluate customer interactions in detail. For example, keywords from customer service are extracted and the flow of the conversation is logically structured, and an evaluation output is generated.

[0366] Step 3:

[0367] The server uses an emotion analysis engine to analyze the tone of voice and language patterns of employees. Input data includes voice and text information, and the analysis results in the output of the employee's emotional state, such as stress level and tension level. These results are then used in the subsequent feedback generation process.

[0368] Step 4:

[0369] The server generates real-time feedback based on natural language processing and sentiment analysis results. This feedback includes specific improvement advice and encouraging messages regarding customer service methods. The generated feedback is sent to the terminal, and the user receives it and uses it as information to quickly correct their response.

[0370] Step 5:

[0371] The server continuously monitors each employee's working hours and workload, and automatically generates optimized work shifts. It uses operational performance and workload data as input, and provides an efficient schedule plan as output. This reduces the burden on employees and improves operational efficiency.

[0372] Step 6:

[0373] Finally, the server combines each employee's performance evaluation and emotional data to propose a personalized training program. Past evaluation history and emotional states are used as input data, and a personalized training plan is generated as output. This plan is provided to the user as e-learning materials or in-person sessions to support skill improvement.

[0374] (Application Example 2)

[0375] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0376] Traditional human resource management systems often fail to adequately consider the emotional state of sales staff and their interactions with customers, making it difficult to optimize operational efficiency and individual growth. Therefore, there is a need for technologies that can improve the quality of customer service, reduce staff stress, and provide effective training.

[0377] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0378] In this invention, the server includes means for collecting the status of members involved in sales operations and sales data; means for centrally managing the collected information and storing it in a storage device that can be accessed in real time; means for analyzing customer service content and generating evaluations using natural language processing technology; means for providing immediate feedback to members based on the analysis results; means for analyzing the status of customers and members using emotional data from devices worn by members; means for automatically optimizing work schedules considering working hours and workload; and means for proposing training plans based on individual work evaluations. This enables flexible work operations that take into account the emotional state of the crew, improved quality of customer service, and effective training based on individual needs.

[0379] "Members involved in sales operations" refers to individuals who are responsible for providing goods and services at stores or sales locations.

[0380] "Sales data" refers to data that includes information related to sales operations, such as product sales, customer information, and purchase history.

[0381] "Centralized management" refers to maintaining a state where information and data are centrally managed and easily accessible as needed.

[0382] A "real-time accessible storage device" refers to a data storage system where information is updated instantly and can be accessed immediately as needed.

[0383] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0384] "Analyzing customer service content and generating evaluations" refers to the process of thoroughly investigating the content of customer interactions and creating evaluations based on the results.

[0385] "Using emotional data to analyze the state of customers and team members" refers to evaluating the mental state of customers and crew members using emotional data obtained from facial expressions, voice, and other sources.

[0386] "Automatically optimizing work schedules" refers to the process of automatically creating the optimal work schedule by taking into account working hours and workload.

[0387] "Proposing a training plan based on individual performance evaluations" refers to the act of proposing a training program to support the growth of each team member based on an evaluation of their work performance.

[0388] In the system realizing this invention, the server utilizes various devices and software to manage and analyze information obtained from sales activities in which members participate. Specifically, the server collects sales data and sentiment data recorded by members during their work and stores them centrally in a storage device. This enables real-time data access.

[0389] The terminals refer to smart glasses and smartphones worn by crew members, which acquire video and audio from these devices and transmit them to a server. The server analyzes this data using natural language processing technology to evaluate the quality of customer service. The analysis results are immediately fed back to the team members, who then provide specific suggestions for improving customer service methods and reducing stress.

[0390] The server utilizes an emotion engine to analyze the emotional states of crew members and customers in real time. This allows it to provide useful information for improving the quality of customer service. For example, if a crew member is feeling stressed, the server will suggest relaxation techniques. The server also has the ability to estimate workload and automatically optimize work schedules based on this.

[0391] For example, if this system were used in a department store's physical location, schedules would be adjusted to allow crew members to take appropriate breaks even during busy periods, and a phase would be presented where additional information about expensive items is provided when a customer shows interest in them.

[0392] An example of a prompt using a generative AI model would be: "Please explain how you alleviate customer anxiety during today's customer service. Also, if a crew member is feeling stressed during work, how would you help them relax?"

[0393] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0394] Step 1:

[0395] The server receives sales data and audio recordings of customer conversations transmitted from the terminal. The input data consists of information collected during the sales process, which the server centrally stores in storage, enabling real-time access.

[0396] Step 2:

[0397] The server uses natural language processing technology to analyze stored conversation data. The input data is conversational information in audio format, which the server converts to text and performs analysis to evaluate the quality of customer service. As output, it generates evaluation data regarding the quality of customer service. Specifically, it analyzes keywords and tone in the conversation and quantifies customer responses.

[0398] Step 3:

[0399] The server sends real-time feedback to the crew based on the evaluation results. The input here is the evaluated customer service, and the server uses this to generate specific areas for improvement and customer service advice, which is then output to the terminal. This feedback includes encouragement and suggestions for improvement.

[0400] Step 4:

[0401] The terminal transmits video data obtained from smart glasses worn by the crew to the emotion engine. The video data is provided as input, and the emotion engine analyzes the facial expressions of the crew and customers to determine their emotional state. The output is data indicating the emotional state. This enables real-time emotional assessment.

[0402] Step 5:

[0403] After receiving emotional data, the server provides the crew with specific feedback and action plans tailored to their emotional state. The input is data indicating emotional state, and the server generates suggestions such as relaxation techniques and adjustments to customer service style, which are then output to the terminal. Specifically, this includes advice on stress level adjustments and customer service tone.

[0404] Step 6:

[0405] The server aggregates individual crew members' performance evaluations and emotional data to optimize work schedules. It uses member work data and emotional data as input to calculate appropriate working hours and break times. The output provides an optimized work schedule, specifically adjusting it to account for workload and peak times.

[0406] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0407] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0408] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0409] [Third Embodiment]

[0410] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0411] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0412] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0413] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0414] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0415] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0416] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0417] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0418] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0419] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0420] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0421] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0422] To implement this invention, it is necessary to build a system for efficiently collecting and analyzing personnel work data in sales operations and for providing optimal feedback and personnel training. This system includes a server, terminals, and user interfaces.

[0423] In this system, each terminal first records sales data, customer service conversations, and shift information generated during the crew's work in real time. The terminals periodically send this information to a server, which stores the received data in a cloud database. This centralized management prevents data dispersion and duplication, ensuring that the latest information is always available.

[0424] Furthermore, the server uses natural language processing technology to thoroughly analyze the text data of customer interactions. This allows for the evaluation of customer service and the measurement of communication skills and service quality. The analysis results are provided as feedback from the server to each crew member's terminal, which users can then use to improve their daily work. The feedback also includes specific areas for improvement and success stories, thus promoting the growth of the crew members.

[0425] The server also automatically optimizes shifts by considering each crew member's working hours and workload. This maximizes the efficiency of staffing in store operations and helps to equalize working hours. For example, prioritizing the deployment of highly capable crew members during peak hours can improve customer satisfaction.

[0426] In addition, the server utilizes Google's AI technology to generate and propose training programs based on each individual's work performance. For example, if a crew member is analyzed as lacking knowledge of a particular product, the server will recommend an e-learning course to supplement that knowledge, which the user can then use for self-study.

[0427] In this way, by providing efficient data collection, analysis, and feedback in sales operations, as well as an optimal training platform, it is possible to build a system that improves overall operational efficiency and fosters the growth of individual crew members.

[0428] The following describes the processing flow.

[0429] Step 1:

[0430] The terminal records sales data entered by crew members during sales activities, customer conversations, and shift information in real time, and periodically sends this information to the server.

[0431] Step 2:

[0432] The server stores the collected data in a cloud database. During this process, data cleansing is performed to remove incomplete or duplicate data and organize it into a unified set of information.

[0433] Step 3:

[0434] The server uses natural language processing technology to analyze text data of customer interactions. This allows for the evaluation of customer service and the measurement of communication skills and service quality.

[0435] Step 4:

[0436] Based on the analysis results, the server generates feedback including areas for improvement and success stories, and provides it to each crew member's terminal in real time. Users then use this feedback to improve their work processes.

[0437] Step 5:

[0438] The server uses IBM's optimization engine to automatically optimize shifts based on crew working hours and workload. This enables efficient staffing.

[0439] Step 6:

[0440] The server analyzes each crew member's work performance data and automatically generates the optimal training program based on that data. Leveraging Google's AI technology, the server proposes specific training and learning plans to the user.

[0441] Step 7:

[0442] Users can access training programs and feedback provided through their devices to aid in self-study and skill development. Each crew member uses this information to improve their performance and contribute to their own growth.

[0443] (Example 1)

[0444] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0445] In sales operations, improving overall operational efficiency while promoting efficient performance enhancement and growth among workers requires the rapid and accurate collection of individual worker activity data and the provision of feedback based on the analysis results. However, conventional methods have made it difficult to achieve this effectively. The main reasons for this include the difficulty of real-time data management, insufficient quality of feedback, and the manual adjustment of working hours and workload.

[0446] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0447] In this invention, the server includes means for acquiring worker activity data in sales operations, means for centrally managing the acquired data and storing it in an information repository for immediate use, and means for analyzing the content of interactions and creating evaluations using natural language processing technology. This enables real-time data management and highly effective feedback based on analysis results, thereby promoting the efficiency and growth of workers.

[0448] "Sales operations" refers to all activities aimed at providing goods and services to customers, and primarily includes commercial transactions and customer service conducted within a store.

[0449] "Workers" refers to individual staff members or employees engaged in sales operations, who, as part of their duties, interact with customers and engage in sales activities.

[0450] "Activity data" refers to all information related to an employee's work performance, including sales information, customer service details, and shift data.

[0451] "Centralized management" refers to a management method that involves consolidating dispersed data into a single information repository for easy access and use.

[0452] An "information storage facility" refers to a database or data storage system used to centrally store collected data.

[0453] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and process the language that humans use in everyday life, and is used for analyzing text and understanding intent.

[0454] "Content of interaction" refers to the content of communication between the worker and the customer, and includes conversations and question-and-answer sessions in customer service.

[0455] "Evaluation" refers to judgments and ratings of workers and their activities obtained based on the analysis results.

[0456] "Immediately available" means that data is ready to be processed and used as soon as it is collected, enabling real-time access.

[0457] To implement this invention, it is necessary to construct a system that efficiently acquires and manages activity data of workers engaged in sales operations, and provides analysis and feedback based on that data. The system includes an internet-connected terminal, a server for data processing, and a user interface for providing information.

[0458] The terminal records activity data in real time, such as sales data, customer service details, and shift information, as employees perform their duties in the store. This utilizes POS systems and voice recognition software. The terminal periodically transmits the collected information to a server via a secure protocol.

[0459] The server centralizes and manages the received data and stores it in a cloud-based information repository. Using advanced natural language processing technology, the server analyzes the text data of customer service interactions and evaluates the quality of customer service. Common natural language APIs are used for this processing. Based on the analysis results, the server generates immediate feedback and provides it to the worker via their terminal. The feedback includes specific areas for improvement and success stories, allowing workers to use it to improve the quality of their work.

[0460] Users receive feedback from the server, which helps them improve their daily work. This process allows workers to continuously improve their work performance.

[0461] As a concrete example, if the server analyzes the content of customer service conversations recorded on a terminal and finds that the response to the customer's request was insufficient, the server will provide feedback with suggestions for improving the conversation. Based on these suggestions, the user can learn how to improve their customer service.

[0462] A useful example of a prompt would be: "We want to analyze customer service conversation data and evaluate customer service. Please tell us how to provide feedback on specific areas for improvement and successful cases."

[0463] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0464] Step 1:

[0465] The terminal acquires sales data, customer service details, and shift information in real time as input while the worker is active. Specifically, it captures transaction data from the POS system and transcribes conversations into text using speech recognition software. As a result, various business data is output in digital format.

[0466] Step 2:

[0467] The terminal encrypts the acquired data and sends it to the server as input using a secure communication protocol. Specifically, this involves data transmission using SSL / TLS. The data is then securely delivered to the server as output.

[0468] Step 3:

[0469] The server stores the received data as input in a cloud-based information repository. During this process, data duplication checks and format conversions are performed, and a centralized database is constructed as output.

[0470] Step 4:

[0471] The server extracts text data of customer service interactions from a cloud-based database and analyzes it using natural language processing technology. Specifically, it performs keyword extraction and sentiment analysis to generate customer service evaluations. The results of the analysis are output as evaluation scores and recommended areas for improvement.

[0472] Step 5:

[0473] The server generates feedback as input based on the analysis results. Specifically, it creates reports that clearly indicate areas for improvement and guidelines that incorporate successful case studies. This is then sent as output to the terminal, notifying the worker.

[0474] Step 6:

[0475] Users receive feedback via their devices as input and utilize it to improve their work. Specifically, they use this information to review their daily customer interactions, reflect it in self-evaluations and team meetings, and use it as output to improve their skills.

[0476] (Application Example 1)

[0477] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0478] In the current sales environment, it is difficult to efficiently utilize information on the behavior of human resources and the content of customer interactions to provide appropriate feedback on the spot and improve the quality of sales. As a result, progress in operational efficiency and improvement of customer service may not be made, potentially leading to a decline in competitiveness. Furthermore, providing effective training plans requires a great deal of time and effort, posing challenges to the growth of individual human resources.

[0479] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0480] In this invention, the server includes means for collecting behavioral information of human resources in sales operations, means for instantly displaying customer and product information when human resources are performing sales activities using a visual assistance device, and means for analyzing customer interactions through speech recognition and language analysis and providing information based on the results. This makes it possible to immediately provide information tailored to the customer and improve the quality of sales. Furthermore, it is possible to effectively support the growth of human resources through the proposal of training plans based on individual work results.

[0481] "Sales activities" refer to a series of activities or processes for providing goods or services to customers.

[0482] "Human resources" refers to the personnel necessary for an organization or company to utilize people's work abilities and experience.

[0483] "Behavioral information" refers to data and records concerning the actions and activities of an individual or group.

[0484] "Visual assistance devices" refer to technological devices that present information to users visually, and smart glasses are an example of such devices.

[0485] "Speech recognition" is a technical process that acquires speech data and converts it into text or commands.

[0486] "Linguistic analysis" is a technique for extracting and interpreting information contained in natural language.

[0487] An "educational plan" is an organized set of learning content and methods aimed at acquiring specific knowledge and skills.

[0488] A "database" is an information system that organizes and manages a collection of related data.

[0489] In this embodiment of the invention, a server, a terminal, and a user cooperate to form a system. The server has the function of collecting behavioral information of human resources in sales operations and storing it in a centrally managed storage medium. This information is accessible in real time, and the server processes the information immediately and generates evaluations.

[0490] The terminal functions as a visual aid, displaying customer and product information within the user's field of vision during sales activities. This allows the user to provide appropriate product suggestions and services based on the content of their conversations with customers. Using speech recognition technology to transcribe customer conversations into text and language analysis technology to interpret the content, the server provides accurate feedback to the user.

[0491] As a concrete example, if a user is assigned to a bookstore and a customer is looking for a novel of a specific genre, the terminal analyzes the customer's speech and displays relevant book information in real time. This allows the customer to quickly find the product they are looking for.

[0492] Furthermore, this system also includes a function to propose educational plans based on individual performance outcomes. The server analyzes each user's work data and automatically generates educational plans to improve specific skills and knowledge as needed. Users can then utilize these plans for self-improvement.

[0493] Example prompt: "A customer came into the bookstore and said they like mystery novels. Please select a book to recommend and provide some customer service tips."

[0494] This system is expected to create an environment that supports increased efficiency in sales operations and the growth of human resources.

[0495] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0496] Step 1:

[0497] The server receives information on the behavior of human resources and the content of customer interactions as audio data from the terminal. Using this audio data as input, the server uses the Google Speech-to-Text API to convert it from audio to text data. The resulting text data is then output.

[0498] Step 2:

[0499] The server uses the Google Cloud Natural Language API to perform language analysis on the text data obtained in Step 1. Here, customer needs are extracted from the text data, and related information is analyzed. The analysis results output information about customer needs.

[0500] Step 3:

[0501] The server compares the analysis results from step 2 with the product information stored on the storage medium and generates a list of products that meet the customer's needs. This product list is output and sent to the terminal, where it is displayed on the user's visual terminal.

[0502] Step 4:

[0503] Based on product information displayed on a visual terminal, users suggest recommended products to customers. Some of the user's actions are recorded again as behavioral data and used for future feedback.

[0504] Step 5:

[0505] The server uses a generative AI model, taking user work data and past feedback as input, to automatically generate training plans tailored to the user's work performance. This training plan is then output and presented to the user via their terminal.

[0506] By efficiently processing data at each step and providing feedback to users, the overall quality of sales operations can be improved.

[0507] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0508] This invention is an efficient human resource management system for sales operations that incorporates an emotion engine. It promotes operational efficiency and crew growth through the collection and analysis of operational data, appropriate feedback, and the proposal of training programs. The emotion engine allows for the recognition of users' emotional states and their reflection in their work.

[0509] The terminal records sales information, customer conversations, and work shift data entered by crew members during sales activities in real time. This data is sent to a server and centrally managed. The server analyzes the collected data using natural language processing technology to evaluate the quality of customer service and communication skills.

[0510] Furthermore, the server is equipped with an emotion engine that can detect in real time the emotions and stress levels that crew members exhibit while serving customers. For example, if the emotion engine detects that a crew member's stress levels are rising based on their facial expressions or tone of voice, the analysis results are reflected in the server, and appropriate feedback is provided.

[0511] This feedback is sent to each crew member's terminal, allowing users to adjust their customer service style and approach according to their own emotional state. For example, if the emotion engine detects tension during a crew member's interaction with a customer, the server provides feedback such as suggestions for relaxation techniques or the next point of focus.

[0512] Furthermore, the server automatically generates training programs tailored to individual needs based on each crew member's performance evaluation and emotional data. These training programs are delivered in the form of e-learning courses and in-person sessions, serving as a guide for users' growth.

[0513] In this way, this system, which incorporates an emotional engine, understands the emotional state of the crew and supports work efficiency and individual growth through appropriate feedback and training.

[0514] The following describes the processing flow.

[0515] Step 1:

[0516] The terminal records sales information and customer conversations entered by the crew during sales activities in real time, and transmits this information to the server.

[0517] Step 2:

[0518] The server receives data sent from terminals and centrally manages it in a cloud database. It also performs data cleansing and organizes any data that is corrupted.

[0519] Step 3:

[0520] The server uses natural language processing technology to analyze customer service interactions and evaluate the quality of service and communication skills. The analysis results serve as the basis for feedback.

[0521] Step 4:

[0522] The server has an emotion engine built in that detects the emotional state of the crew in real time from their tone of voice and facial expressions. It identifies when they are emotionally agitated or under high stress.

[0523] Step 5:

[0524] The server generates immediate feedback based on the analyzed sentiment data and customer service evaluation, and sends it to the user's device. This allows the user to understand areas for improvement in their customer service style.

[0525] Step 6:

[0526] Users who see feedback that includes data from the emotion engine can use that feedback to adjust their customer service methods. For example, they might be mindful of relaxation techniques or points to keep in mind for future interactions.

[0527] Step 7:

[0528] The server automatically generates training programs to promote growth based on each crew member's performance evaluation and emotional data. The generated programs are then provided to users via their terminals.

[0529] Step 8:

[0530] Users aim to improve their skills and obtain certifications by following training programs provided on their devices. Through this process, users can continuously improve themselves.

[0531] (Example 2)

[0532] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0533] To improve the operational efficiency of personnel in sales operations, it is necessary not only to collect operational data, but also to accurately grasp the actual performance and emotional state of individual employees, provide immediate feedback, and offer appropriate training. However, conventional systems do not integrate these elements, which has led to challenges such as decreased operational efficiency and stifled opportunities for individual growth.

[0534] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0535] In this invention, the server includes means for collecting work information related to sales operations, means for centrally controlling the collected information and storing it in an information storage facility that can be accessed immediately, means for analyzing customer interaction content and emotional state and generating evaluations using natural language processing and sentiment analysis engines, means for providing responses in real time based on the analysis results and sentiment analysis results, means for automatically optimizing work shifts considering working hours and workload, and means for providing educational programs based on each individual's work evaluation and emotional state. This makes it possible to provide feedback and educational programs from both work data and emotional perspectives, thereby simultaneously achieving improved work efficiency and employee growth.

[0536] "Sales operations" refer to a series of activities related to the provision and transaction of goods, and include operations that involve direct interaction with customers.

[0537] "Work information" refers to data about the specific activities performed by employees in a business process and the results thereof.

[0538] "Centralized control" refers to a state where collected data is centrally managed through a central system or platform, and access and processing are integrated.

[0539] An "information storage facility" is a storage device or database used to store various types of data and make them readily accessible as needed.

[0540] "Natural language processing" refers to the technology of processing human language using computers and interpreting its meaning and intent.

[0541] An "emotion analysis engine" is a system that recognizes a person's emotional state from their voice, facial expressions, and other information, and performs analysis based on that information.

[0542] "Generating an evaluation" means quantifying or judging the performance of an individual or behavior according to specific criteria, based on collected data.

[0543] "Providing real-time responses" means providing feedback and suggestions for improvement almost immediately based on collected and analyzed data.

[0544] "Automatically optimizing work shifts based on working hours and workload" is a process that automatically readjusts schedules to efficiently adjust employee work effort and time management, thereby improving labor efficiency.

[0545] An "educational program" is a general term for training and guidance activities aimed at improving employees' skills and knowledge.

[0546] This invention is a system that promotes the work efficiency and individual growth of employees in sales operations. Specifically, it is achieved by collecting work information from the sales floor in real time and using that information for evaluation and feedback.

[0547] The terminals collect sales information and customer conversation details using voice input and touch controls while each employee is performing sales duties. This data is immediately transmitted to the server.

[0548] The server stores the collected data in an information repository and controls it centrally. Furthermore, the server is equipped with natural language processing software and an emotion analysis engine to analyze the transmitted data. This analysis generates evaluations of customer service and the emotional states of employees.

[0549] The server has the ability to provide real-time feedback to each employee based on the analysis results. This feedback is delivered immediately to the terminal and includes specific advice, such as, "Your voice is trembling, so try speaking more slowly to calm yourself down."

[0550] Furthermore, the server automatically proposes optimal shifts, taking into account employee working hours and workload. Based on work evaluations and sentiment analysis, it also suggests training programs that will maximize the benefits for each employee. These programs are delivered in e-learning or in-person session formats.

[0551] For example, when a cafe employee receives an order for "two cafe lattes" at the counter, real-time feedback such as "It's busy today, so please try to work efficiently" is provided. Another example of a prompt message is: "Generate feedback to improve customer service skills in the cafe. Situation: Employee is stressed. Needed feedback: Relaxation techniques and the next steps in customer service."

[0552] Thus, the present invention is a system that simultaneously achieves work efficiency and personal growth by integrating and analyzing work data and emotional states, and providing corresponding feedback and educational programs.

[0553] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0554] Step 1:

[0555] The terminal acquires sales information and customer interaction transcripts entered by employees during sales activities. This information is collected through touch input and voice recognition. Input data includes specific product names, quantities, and transcripts of customer conversations. The collected information is transmitted to the server in real time.

[0556] Step 2:

[0557] The server receives data sent from the terminal and stores it in the information repository. Next, natural language processing technology is used to convert the content of customer interactions into structured data. This data processing makes it possible to evaluate customer interactions in detail. For example, keywords from customer service are extracted and the flow of the conversation is logically structured, and an evaluation output is generated.

[0558] Step 3:

[0559] The server uses an emotion analysis engine to analyze the tone of voice and language patterns of employees. Input data includes voice and text information, and the analysis results in the output of the employee's emotional state, such as stress level and tension level. These results are then used in the subsequent feedback generation process.

[0560] Step 4:

[0561] The server generates real-time feedback based on natural language processing and sentiment analysis results. This feedback includes specific improvement advice and encouraging messages regarding customer service methods. The generated feedback is sent to the terminal, and the user receives it and uses it as information to quickly correct their response.

[0562] Step 5:

[0563] The server continuously monitors each employee's working hours and workload, and automatically generates optimized work shifts. It uses operational performance and workload data as input, and provides an efficient schedule plan as output. This reduces the burden on employees and improves operational efficiency.

[0564] Step 6:

[0565] Finally, the server combines each employee's performance evaluation and emotional data to propose a personalized training program. Past evaluation history and emotional states are used as input data, and a personalized training plan is generated as output. This plan is provided to the user as e-learning materials or in-person sessions to support skill improvement.

[0566] (Application Example 2)

[0567] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0568] Traditional human resource management systems often fail to adequately consider the emotional state of sales staff and their interactions with customers, making it difficult to optimize operational efficiency and individual growth. Therefore, there is a need for technologies that can improve the quality of customer service, reduce staff stress, and provide effective training.

[0569] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0570] In this invention, the server includes means for collecting the status of members involved in sales operations and sales data; means for centrally managing the collected information and storing it in a storage device that can be accessed in real time; means for analyzing customer service content and generating evaluations using natural language processing technology; means for providing immediate feedback to members based on the analysis results; means for analyzing the status of customers and members using emotional data from devices worn by members; means for automatically optimizing work schedules considering working hours and workload; and means for proposing training plans based on individual work evaluations. This enables flexible work operations that take into account the emotional state of the crew, improved quality of customer service, and effective training based on individual needs.

[0571] "Members involved in sales operations" refers to individuals who are responsible for providing goods and services at stores or sales locations.

[0572] "Sales data" refers to data that includes information related to sales operations, such as product sales, customer information, and purchase history.

[0573] "Centralized management" refers to maintaining a state where information and data are centrally managed and easily accessible as needed.

[0574] A "real-time accessible storage device" refers to a data storage system where information is updated instantly and can be accessed immediately as needed.

[0575] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0576] "Analyzing customer service content and generating evaluations" refers to the process of thoroughly investigating the content of customer interactions and creating evaluations based on the results.

[0577] "Using emotional data to analyze the state of customers and team members" refers to evaluating the mental state of customers and crew members using emotional data obtained from facial expressions, voice, and other sources.

[0578] "Automatically optimizing work schedules" refers to the process of automatically creating the optimal work schedule by taking into account working hours and workload.

[0579] "Proposing a training plan based on individual performance evaluations" refers to the act of proposing a training program to support the growth of each team member based on an evaluation of their work performance.

[0580] In the system realizing this invention, the server utilizes various devices and software to manage and analyze information obtained from sales activities in which members participate. Specifically, the server collects sales data and sentiment data recorded by members during their work and stores them centrally in a storage device. This enables real-time data access.

[0581] The terminals refer to smart glasses and smartphones worn by crew members, which acquire video and audio from these devices and transmit them to a server. The server analyzes this data using natural language processing technology to evaluate the quality of customer service. The analysis results are immediately fed back to the team members, who then provide specific suggestions for improving customer service methods and reducing stress.

[0582] The server utilizes an emotion engine to analyze the emotional states of crew members and customers in real time. This allows it to provide useful information for improving the quality of customer service. For example, if a crew member is feeling stressed, the server will suggest relaxation techniques. The server also has the ability to estimate workload and automatically optimize work schedules based on this.

[0583] For example, if this system were used in a department store's physical location, schedules would be adjusted to allow crew members to take appropriate breaks even during busy periods, and a phase would be presented where additional information about expensive items is provided when a customer shows interest in them.

[0584] An example of a prompt using a generative AI model would be: "Please explain how you alleviate customer anxiety during today's customer service. Also, if a crew member is feeling stressed during work, how would you help them relax?"

[0585] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0586] Step 1:

[0587] The server receives sales data and audio recordings of customer conversations transmitted from the terminal. The input data consists of information collected during the sales process, which the server centrally stores in storage, enabling real-time access.

[0588] Step 2:

[0589] The server uses natural language processing technology to analyze stored conversation data. The input data is conversational information in audio format, which the server converts to text and performs analysis to evaluate the quality of customer service. As output, it generates evaluation data regarding the quality of customer service. Specifically, it analyzes keywords and tone in the conversation and quantifies customer responses.

[0590] Step 3:

[0591] The server sends real-time feedback to the crew based on the evaluation results. The input here is the evaluated customer service, and the server uses this to generate specific areas for improvement and customer service advice, which is then output to the terminal. This feedback includes encouragement and suggestions for improvement.

[0592] Step 4:

[0593] The terminal transmits video data obtained from smart glasses worn by the crew to the emotion engine. The video data is provided as input, and the emotion engine analyzes the facial expressions of the crew and customers to determine their emotional state. The output is data indicating the emotional state. This enables real-time emotional assessment.

[0594] Step 5:

[0595] After receiving emotional data, the server provides the crew with specific feedback and action plans tailored to their emotional state. The input is data indicating emotional state, and the server generates suggestions such as relaxation techniques and adjustments to customer service style, which are then output to the terminal. Specifically, this includes advice on stress level adjustments and customer service tone.

[0596] Step 6:

[0597] The server aggregates individual crew members' performance evaluations and emotional data to optimize work schedules. It uses member work data and emotional data as input to calculate appropriate working hours and break times. The output provides an optimized work schedule, specifically adjusting it to account for workload and peak times.

[0598] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0599] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0600] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0601] [Fourth Embodiment]

[0602] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0603] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0604] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0605] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0606] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0607] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0608] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0609] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0610] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0611] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0612] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0613] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0614] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0615] To implement this invention, it is necessary to build a system for efficiently collecting and analyzing personnel work data in sales operations and for providing optimal feedback and personnel training. This system includes a server, terminals, and user interfaces.

[0616] In this system, each terminal first records sales data, customer service conversations, and shift information generated during the crew's work in real time. The terminals periodically send this information to a server, which stores the received data in a cloud database. This centralized management prevents data dispersion and duplication, ensuring that the latest information is always available.

[0617] Furthermore, the server uses natural language processing technology to thoroughly analyze the text data of customer interactions. This allows for the evaluation of customer service and the measurement of communication skills and service quality. The analysis results are provided as feedback from the server to each crew member's terminal, which users can then use to improve their daily work. The feedback also includes specific areas for improvement and success stories, thus promoting the growth of the crew members.

[0618] The server also automatically optimizes shifts by considering each crew member's working hours and workload. This maximizes the efficiency of staffing in store operations and helps to equalize working hours. For example, prioritizing the deployment of highly capable crew members during peak hours can improve customer satisfaction.

[0619] In addition, the server utilizes Google's AI technology to generate and propose training programs based on each individual's work performance. For example, if a crew member is analyzed as lacking knowledge of a particular product, the server will recommend an e-learning course to supplement that knowledge, which the user can then use for self-study.

[0620] In this way, by providing efficient data collection, analysis, and feedback in sales operations, as well as an optimal training platform, it is possible to build a system that improves overall operational efficiency and fosters the growth of individual crew members.

[0621] The following describes the processing flow.

[0622] Step 1:

[0623] The terminal records sales data entered by crew members during sales activities, customer conversations, and shift information in real time, and periodically sends this information to the server.

[0624] Step 2:

[0625] The server stores the collected data in a cloud database. During this process, data cleansing is performed to remove incomplete or duplicate data and organize it into a unified set of information.

[0626] Step 3:

[0627] The server uses natural language processing technology to analyze text data of customer interactions. This allows for the evaluation of customer service and the measurement of communication skills and service quality.

[0628] Step 4:

[0629] Based on the analysis results, the server generates feedback including areas for improvement and success stories, and provides it to each crew member's terminal in real time. Users then use this feedback to improve their work processes.

[0630] Step 5:

[0631] The server uses IBM's optimization engine to automatically optimize shifts based on crew working hours and workload. This enables efficient staffing.

[0632] Step 6:

[0633] The server analyzes each crew member's work performance data and automatically generates the optimal training program based on that data. Leveraging Google's AI technology, the server proposes specific training and learning plans to the user.

[0634] Step 7:

[0635] Users can access training programs and feedback provided through their devices to aid in self-study and skill development. Each crew member uses this information to improve their performance and contribute to their own growth.

[0636] (Example 1)

[0637] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0638] In sales operations, improving overall operational efficiency while promoting efficient performance enhancement and growth among workers requires the rapid and accurate collection of individual worker activity data and the provision of feedback based on the analysis results. However, conventional methods have made it difficult to achieve this effectively. The main reasons for this include the difficulty of real-time data management, insufficient quality of feedback, and the manual adjustment of working hours and workload.

[0639] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0640] In this invention, the server includes means for acquiring worker activity data in sales operations, means for centrally managing the acquired data and storing it in an information repository for immediate use, and means for analyzing the content of interactions and creating evaluations using natural language processing technology. This enables real-time data management and highly effective feedback based on analysis results, thereby promoting the efficiency and growth of workers.

[0641] "Sales operations" refers to all activities aimed at providing goods and services to customers, and primarily includes commercial transactions and customer service conducted within a store.

[0642] "Workers" refers to individual staff members or employees engaged in sales operations, who, as part of their duties, interact with customers and engage in sales activities.

[0643] "Activity data" refers to all information related to an employee's work performance, including sales information, customer service details, and shift data.

[0644] "Centralized management" refers to a management method that involves consolidating dispersed data into a single information repository for easy access and use.

[0645] An "information storage facility" refers to a database or data storage system used to centrally store collected data.

[0646] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and process the language that humans use in everyday life, and is used for analyzing text and understanding intent.

[0647] "Content of interaction" refers to the content of communication between the worker and the customer, and includes conversations and question-and-answer sessions in customer service.

[0648] "Evaluation" refers to judgments and ratings of workers and their activities obtained based on the analysis results.

[0649] "Immediately available" means that data is ready to be processed and used as soon as it is collected, enabling real-time access.

[0650] To implement this invention, it is necessary to construct a system that efficiently acquires and manages activity data of workers engaged in sales operations, and provides analysis and feedback based on that data. The system includes an internet-connected terminal, a server for data processing, and a user interface for providing information.

[0651] The terminal records activity data in real time, such as sales data, customer service details, and shift information, as employees perform their duties in the store. This utilizes POS systems and voice recognition software. The terminal periodically transmits the collected information to a server via a secure protocol.

[0652] The server centralizes and manages the received data and stores it in a cloud-based information repository. Using advanced natural language processing technology, the server analyzes the text data of customer service interactions and evaluates the quality of customer service. Common natural language APIs are used for this processing. Based on the analysis results, the server generates immediate feedback and provides it to the worker via their terminal. The feedback includes specific areas for improvement and success stories, allowing workers to use it to improve the quality of their work.

[0653] Users receive feedback from the server, which helps them improve their daily work. This process allows workers to continuously improve their work performance.

[0654] As a concrete example, if the server analyzes the content of customer service conversations recorded on a terminal and finds that the response to the customer's request was insufficient, the server will provide feedback with suggestions for improving the conversation. Based on these suggestions, the user can learn how to improve their customer service.

[0655] A useful example of a prompt would be: "We want to analyze customer service conversation data and evaluate customer service. Please tell us how to provide feedback on specific areas for improvement and successful cases."

[0656] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0657] Step 1:

[0658] The terminal acquires sales data, customer service details, and shift information in real time as input while the worker is active. Specifically, it captures transaction data from the POS system and transcribes conversations into text using speech recognition software. As a result, various business data is output in digital format.

[0659] Step 2:

[0660] The terminal encrypts the acquired data and sends it to the server as input using a secure communication protocol. Specifically, this involves data transmission using SSL / TLS. The data is then securely delivered to the server as output.

[0661] Step 3:

[0662] The server stores the received data as input in a cloud-based information repository. During this process, data duplication checks and format conversions are performed, and a centralized database is constructed as output.

[0663] Step 4:

[0664] The server extracts text data of customer service interactions from a cloud-based database and analyzes it using natural language processing technology. Specifically, it performs keyword extraction and sentiment analysis to generate customer service evaluations. The results of the analysis are output as evaluation scores and recommended areas for improvement.

[0665] Step 5:

[0666] The server generates feedback as input based on the analysis results. Specifically, it creates reports that clearly indicate areas for improvement and guidelines that incorporate successful case studies. This is then sent as output to the terminal, notifying the worker.

[0667] Step 6:

[0668] Users receive feedback via their devices as input and utilize it to improve their work. Specifically, they use this information to review their daily customer interactions, reflect it in self-evaluations and team meetings, and use it as output to improve their skills.

[0669] (Application Example 1)

[0670] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0671] In the current sales environment, it is difficult to efficiently utilize information on the behavior of human resources and the content of customer interactions to provide appropriate feedback on the spot and improve the quality of sales. As a result, progress in operational efficiency and improvement of customer service may not be made, potentially leading to a decline in competitiveness. Furthermore, providing effective training plans requires a great deal of time and effort, posing challenges to the growth of individual human resources.

[0672] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0673] In this invention, the server includes means for collecting behavioral information of human resources in sales operations, means for instantly displaying customer and product information when human resources are performing sales activities using a visual assistance device, and means for analyzing customer interactions through speech recognition and language analysis and providing information based on the results. This makes it possible to immediately provide information tailored to the customer and improve the quality of sales. Furthermore, it is possible to effectively support the growth of human resources through the proposal of training plans based on individual work results.

[0674] "Sales activities" refer to a series of activities or processes for providing goods or services to customers.

[0675] "Human resources" refers to the personnel necessary for an organization or company to utilize people's work abilities and experience.

[0676] "Behavioral information" refers to data and records concerning the actions and activities of an individual or group.

[0677] "Visual assistance devices" refer to technological devices that present information to users visually, and smart glasses are an example of such devices.

[0678] "Speech recognition" is a technical process that acquires speech data and converts it into text or commands.

[0679] "Linguistic analysis" is a technique for extracting and interpreting information contained in natural language.

[0680] An "educational plan" is an organized set of learning content and methods aimed at acquiring specific knowledge and skills.

[0681] A "database" is an information system that organizes and manages a collection of related data.

[0682] In this embodiment of the invention, a server, a terminal, and a user cooperate to form a system. The server has the function of collecting behavioral information of human resources in sales operations and storing it in a centrally managed storage medium. This information is accessible in real time, and the server processes the information immediately and generates evaluations.

[0683] The terminal functions as a visual aid, displaying customer and product information within the user's field of vision during sales activities. This allows the user to provide appropriate product suggestions and services based on the content of their conversations with customers. Using speech recognition technology to transcribe customer conversations into text and language analysis technology to interpret the content, the server provides accurate feedback to the user.

[0684] As a concrete example, if a user is assigned to a bookstore and a customer is looking for a novel of a specific genre, the terminal analyzes the customer's speech and displays relevant book information in real time. This allows the customer to quickly find the product they are looking for.

[0685] Furthermore, this system also includes a function to propose educational plans based on individual performance outcomes. The server analyzes each user's work data and automatically generates educational plans to improve specific skills and knowledge as needed. Users can then utilize these plans for self-improvement.

[0686] Example prompt: "A customer came into the bookstore and said they like mystery novels. Please select a book to recommend and provide some customer service tips."

[0687] This system is expected to create an environment that supports increased efficiency in sales operations and the growth of human resources.

[0688] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0689] Step 1:

[0690] The server receives information on the behavior of human resources and the content of customer interactions as audio data from the terminal. Using this audio data as input, the server uses the Google Speech-to-Text API to convert it from audio to text data. The resulting text data is then output.

[0691] Step 2:

[0692] The server uses the Google Cloud Natural Language API to perform language analysis on the text data obtained in Step 1. Here, customer needs are extracted from the text data, and related information is analyzed. The analysis results output information about customer needs.

[0693] Step 3:

[0694] The server compares the analysis results from step 2 with the product information stored on the storage medium and generates a list of products that meet the customer's needs. This product list is output and sent to the terminal, where it is displayed on the user's visual terminal.

[0695] Step 4:

[0696] Based on product information displayed on a visual terminal, users suggest recommended products to customers. Some of the user's actions are recorded again as behavioral data and used for future feedback.

[0697] Step 5:

[0698] The server uses a generative AI model, taking user work data and past feedback as input, to automatically generate training plans tailored to the user's work performance. This training plan is then output and presented to the user via their terminal.

[0699] By efficiently processing data at each step and providing feedback to users, the overall quality of sales operations can be improved.

[0700] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0701] This invention is an efficient human resource management system for sales operations that incorporates an emotion engine. It promotes operational efficiency and crew growth through the collection and analysis of operational data, appropriate feedback, and the proposal of training programs. The emotion engine allows for the recognition of users' emotional states and their reflection in their work.

[0702] The terminal records sales information, customer conversations, and work shift data entered by crew members during sales activities in real time. This data is sent to a server and centrally managed. The server analyzes the collected data using natural language processing technology to evaluate the quality of customer service and communication skills.

[0703] Furthermore, the server is equipped with an emotion engine that can detect in real time the emotions and stress levels that crew members exhibit while serving customers. For example, if the emotion engine detects that a crew member's stress levels are rising based on their facial expressions or tone of voice, the analysis results are reflected in the server, and appropriate feedback is provided.

[0704] This feedback is sent to each crew member's terminal, allowing users to adjust their customer service style and approach according to their own emotional state. For example, if the emotion engine detects tension during a crew member's interaction with a customer, the server provides feedback such as suggestions for relaxation techniques or the next point of focus.

[0705] Furthermore, the server automatically generates training programs tailored to individual needs based on each crew member's performance evaluation and emotional data. These training programs are delivered in the form of e-learning courses and in-person sessions, serving as a guide for users' growth.

[0706] In this way, this system, which incorporates an emotional engine, understands the emotional state of the crew and supports work efficiency and individual growth through appropriate feedback and training.

[0707] The following describes the processing flow.

[0708] Step 1:

[0709] The terminal records sales information and customer conversations entered by the crew during sales activities in real time, and transmits this information to the server.

[0710] Step 2:

[0711] The server receives data sent from terminals and centrally manages it in a cloud database. It also performs data cleansing and organizes any data that is corrupted.

[0712] Step 3:

[0713] The server uses natural language processing technology to analyze customer service interactions and evaluate the quality of service and communication skills. The analysis results serve as the basis for feedback.

[0714] Step 4:

[0715] The server has an emotion engine built in that detects the emotional state of the crew in real time from their tone of voice and facial expressions. It identifies when they are emotionally agitated or under high stress.

[0716] Step 5:

[0717] The server generates immediate feedback based on the analyzed sentiment data and customer service evaluation, and sends it to the user's device. This allows the user to understand areas for improvement in their customer service style.

[0718] Step 6:

[0719] Users who see feedback that includes data from the emotion engine can use that feedback to adjust their customer service methods. For example, they might be mindful of relaxation techniques or points to keep in mind for future interactions.

[0720] Step 7:

[0721] The server automatically generates training programs to promote growth based on each crew member's performance evaluation and emotional data. The generated programs are then provided to users via their terminals.

[0722] Step 8:

[0723] Users aim to improve their skills and obtain certifications by following training programs provided on their devices. Through this process, users can continuously improve themselves.

[0724] (Example 2)

[0725] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0726] To improve the operational efficiency of personnel in sales operations, it is necessary not only to collect operational data, but also to accurately grasp the actual performance and emotional state of individual employees, provide immediate feedback, and offer appropriate training. However, conventional systems do not integrate these elements, which has led to challenges such as decreased operational efficiency and stifled opportunities for individual growth.

[0727] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0728] In this invention, the server includes means for collecting work information related to sales operations, means for centrally controlling the collected information and storing it in an information storage facility that can be accessed immediately, means for analyzing customer interaction content and emotional state and generating evaluations using natural language processing and sentiment analysis engines, means for providing responses in real time based on the analysis results and sentiment analysis results, means for automatically optimizing work shifts considering working hours and workload, and means for providing educational programs based on each individual's work evaluation and emotional state. This makes it possible to provide feedback and educational programs from both work data and emotional perspectives, thereby simultaneously achieving improved work efficiency and employee growth.

[0729] "Sales operations" refer to a series of activities related to the provision and transaction of goods, and include operations that involve direct interaction with customers.

[0730] "Work information" refers to data about the specific activities performed by employees in a business process and the results thereof.

[0731] "Centralized control" refers to a state where collected data is centrally managed through a central system or platform, and access and processing are integrated.

[0732] An "information storage facility" is a storage device or database used to store various types of data and make them readily accessible as needed.

[0733] "Natural language processing" refers to the technology of processing human language using computers and interpreting its meaning and intent.

[0734] An "emotion analysis engine" is a system that recognizes a person's emotional state from their voice, facial expressions, and other information, and performs analysis based on that information.

[0735] "Generating an evaluation" means quantifying or judging the performance of an individual or behavior according to specific criteria, based on collected data.

[0736] "Providing real-time responses" means providing feedback and suggestions for improvement almost immediately based on collected and analyzed data.

[0737] "Automatically optimizing work shifts based on working hours and workload" is a process that automatically readjusts schedules to efficiently adjust employee work effort and time management, thereby improving labor efficiency.

[0738] An "educational program" is a general term for training and guidance activities aimed at improving employees' skills and knowledge.

[0739] This invention is a system that promotes the work efficiency and individual growth of employees in sales operations. Specifically, it is achieved by collecting work information from the sales floor in real time and using that information for evaluation and feedback.

[0740] The terminals collect sales information and customer conversation details using voice input and touch controls while each employee is performing sales duties. This data is immediately transmitted to the server.

[0741] The server stores the collected data in an information repository and controls it centrally. Furthermore, the server is equipped with natural language processing software and an emotion analysis engine to analyze the transmitted data. This analysis generates evaluations of customer service and the emotional states of employees.

[0742] The server has the ability to provide real-time feedback to each employee based on the analysis results. This feedback is delivered immediately to the terminal and includes specific advice, such as, "Your voice is trembling, so try speaking more slowly to calm yourself down."

[0743] Furthermore, the server automatically proposes optimal shifts, taking into account employee working hours and workload. Based on work evaluations and sentiment analysis, it also suggests training programs that will maximize the benefits for each employee. These programs are delivered in e-learning or in-person session formats.

[0744] For example, when a cafe employee receives an order for "two cafe lattes" at the counter, real-time feedback such as "It's busy today, so please try to work efficiently" is provided. Another example of a prompt message is: "Generate feedback to improve customer service skills in the cafe. Situation: Employee is stressed. Needed feedback: Relaxation techniques and the next steps in customer service."

[0745] Thus, the present invention is a system that simultaneously achieves work efficiency and personal growth by integrating and analyzing work data and emotional states, and providing corresponding feedback and educational programs.

[0746] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0747] Step 1:

[0748] The terminal acquires sales information and customer interaction transcripts entered by employees during sales activities. This information is collected through touch input and voice recognition. Input data includes specific product names, quantities, and transcripts of customer conversations. The collected information is transmitted to the server in real time.

[0749] Step 2:

[0750] The server receives data sent from the terminal and stores it in the information repository. Next, natural language processing technology is used to convert the content of customer interactions into structured data. This data processing makes it possible to evaluate customer interactions in detail. For example, keywords from customer service are extracted and the flow of the conversation is logically structured, and an evaluation output is generated.

[0751] Step 3:

[0752] The server uses an emotion analysis engine to analyze the tone of voice and language patterns of employees. Input data includes voice and text information, and the analysis results in the output of the employee's emotional state, such as stress level and tension level. These results are then used in the subsequent feedback generation process.

[0753] Step 4:

[0754] The server generates real-time feedback based on natural language processing and sentiment analysis results. This feedback includes specific improvement advice and encouraging messages regarding customer service methods. The generated feedback is sent to the terminal, and the user receives it and uses it as information to quickly correct their response.

[0755] Step 5:

[0756] The server continuously monitors each employee's working hours and workload, and automatically generates optimized work shifts. It uses operational performance and workload data as input, and provides an efficient schedule plan as output. This reduces the burden on employees and improves operational efficiency.

[0757] Step 6:

[0758] Finally, the server combines each employee's performance evaluation and emotional data to propose a personalized training program. Past evaluation history and emotional states are used as input data, and a personalized training plan is generated as output. This plan is provided to the user as e-learning materials or in-person sessions to support skill improvement.

[0759] (Application Example 2)

[0760] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0761] Traditional human resource management systems often fail to adequately consider the emotional state of sales staff and their interactions with customers, making it difficult to optimize operational efficiency and individual growth. Therefore, there is a need for technologies that can improve the quality of customer service, reduce staff stress, and provide effective training.

[0762] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0763] In this invention, the server includes means for collecting the status of members involved in sales operations and sales data; means for centrally managing the collected information and storing it in a storage device that can be accessed in real time; means for analyzing customer service content and generating evaluations using natural language processing technology; means for providing immediate feedback to members based on the analysis results; means for analyzing the status of customers and members using emotional data from devices worn by members; means for automatically optimizing work schedules considering working hours and workload; and means for proposing training plans based on individual work evaluations. This enables flexible work operations that take into account the emotional state of the crew, improved quality of customer service, and effective training based on individual needs.

[0764] "Members involved in sales operations" refers to individuals who are responsible for providing goods and services at stores or sales locations.

[0765] "Sales data" refers to data that includes information related to sales operations, such as product sales, customer information, and purchase history.

[0766] "Centralized management" refers to maintaining a state where information and data are centrally managed and easily accessible as needed.

[0767] A "real-time accessible storage device" refers to a data storage system where information is updated instantly and can be accessed immediately as needed.

[0768] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0769] "Analyzing customer service content and generating evaluations" refers to the process of thoroughly investigating the content of customer interactions and creating evaluations based on the results.

[0770] "Using emotional data to analyze the state of customers and team members" refers to evaluating the mental state of customers and crew members using emotional data obtained from facial expressions, voice, and other sources.

[0771] "Automatically optimizing work schedules" refers to the process of automatically creating the optimal work schedule by taking into account working hours and workload.

[0772] "Proposing a training plan based on individual performance evaluations" refers to the act of proposing a training program to support the growth of each team member based on an evaluation of their work performance.

[0773] In the system realizing this invention, the server utilizes various devices and software to manage and analyze information obtained from sales activities in which members participate. Specifically, the server collects sales data and sentiment data recorded by members during their work and stores them centrally in a storage device. This enables real-time data access.

[0774] The terminals refer to smart glasses and smartphones worn by crew members, which acquire video and audio from these devices and transmit them to a server. The server analyzes this data using natural language processing technology to evaluate the quality of customer service. The analysis results are immediately fed back to the team members, who then provide specific suggestions for improving customer service methods and reducing stress.

[0775] The server utilizes an emotion engine to analyze the emotional states of crew members and customers in real time. This allows it to provide useful information for improving the quality of customer service. For example, if a crew member is feeling stressed, the server will suggest relaxation techniques. The server also has the ability to estimate workload and automatically optimize work schedules based on this.

[0776] For example, if this system were used in a department store's physical location, schedules would be adjusted to allow crew members to take appropriate breaks even during busy periods, and a phase would be presented where additional information about expensive items is provided when a customer shows interest in them.

[0777] An example of a prompt using a generative AI model would be: "Please explain how you alleviate customer anxiety during today's customer service. Also, if a crew member is feeling stressed during work, how would you help them relax?"

[0778] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0779] Step 1:

[0780] The server receives sales data and audio recordings of customer conversations transmitted from the terminal. The input data consists of information collected during the sales process, which the server centrally stores in storage, enabling real-time access.

[0781] Step 2:

[0782] The server uses natural language processing technology to analyze stored conversation data. The input data is conversational information in audio format, which the server converts to text and performs analysis to evaluate the quality of customer service. As output, it generates evaluation data regarding the quality of customer service. Specifically, it analyzes keywords and tone in the conversation and quantifies customer responses.

[0783] Step 3:

[0784] The server sends real-time feedback to the crew based on the evaluation results. The input here is the evaluated customer service, and the server uses this to generate specific areas for improvement and customer service advice, which is then output to the terminal. This feedback includes encouragement and suggestions for improvement.

[0785] Step 4:

[0786] The terminal transmits video data obtained from smart glasses worn by the crew to the emotion engine. The video data is provided as input, and the emotion engine analyzes the facial expressions of the crew and customers to determine their emotional state. The output is data indicating the emotional state. This enables real-time emotional assessment.

[0787] Step 5:

[0788] After receiving emotional data, the server provides the crew with specific feedback and action plans tailored to their emotional state. The input is data indicating emotional state, and the server generates suggestions such as relaxation techniques and adjustments to customer service style, which are then output to the terminal. Specifically, this includes advice on stress level adjustments and customer service tone.

[0789] Step 6:

[0790] The server aggregates individual crew members' performance evaluations and emotional data to optimize work schedules. It uses member work data and emotional data as input to calculate appropriate working hours and break times. The output provides an optimized work schedule, specifically adjusting it to account for workload and peak times.

[0791] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0792] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0793] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0794] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0795] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0796] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0797] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0798] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0799] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0800] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0801] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0802] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0803] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0804] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0805] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0806] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0807] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0808] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0809] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0810] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0811] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0812] The following is further disclosed regarding the embodiments described above.

[0813] (Claim 1)

[0814] A means of collecting personnel work data in sales operations,

[0815] A means of centrally managing the collected data and storing it in a database that can be accessed in real time,

[0816] A means of analyzing customer service content and generating evaluations using natural language processing,

[0817] A means of providing real-time feedback based on the analysis results,

[0818] A method for automatically optimizing shifts by considering working hours and workload,

[0819] A system that includes a means of proposing training programs based on individual work performance.

[0820] (Claim 2)

[0821] The system according to claim 1, comprising means for automatically generating individual training programs based on the results of business data analysis.

[0822] (Claim 3)

[0823] The system according to claim 1, which aims to easily improve performance by using information centrally managed on a database in order to achieve operational efficiency and employee development.

[0824] "Example 1"

[0825] (Claim 1)

[0826] A means of acquiring worker activity data in sales operations,

[0827] A means of centrally managing acquired data and storing it in an information repository that is immediately available,

[0828] A means of analyzing and creating an evaluation of the content of a conversation using natural language processing technology,

[0829] A means of providing immediate feedback based on the analysis results,

[0830] A means of automatically adjusting working hours considering working hours and workload,

[0831] A system that includes means for proposing training programs based on the effectiveness of individual tasks.

[0832] (Claim 2)

[0833] The system according to claim 1, comprising means for automatically generating individual training programs based on the results of analysis of work data.

[0834] (Claim 3)

[0835] The system according to claim 1, which aims to improve work efficiency and worker development by using information centrally managed on an information storage system to easily improve work effectiveness.

[0836] "Application Example 1"

[0837] (Claim 1)

[0838] Means for collecting behavioral information on human resources in sales operations,

[0839] A means of centrally managing the collected information and storing it in a storage medium that allows for immediate retrieval,

[0840] A means of interpreting the content of the interaction and creating an evaluation using language data processing,

[0841] A means of providing immediate reverse feed based on the interpretation result,

[0842] A means to automatically optimize work schedules considering work time and workload,

[0843] A means of proposing an educational plan based on individual behavioral outcomes,

[0844] A means of instantly displaying customer information and product information when human resources conduct sales activities using a visual assistance device,

[0845] A system that includes means of analyzing customer interactions through speech recognition and language analysis, and providing information based on the results.

[0846] (Claim 2)

[0847] The system according to claim 1, comprising means for automatically creating individualized educational plans based on the interpretation results of behavioral information.

[0848] (Claim 3)

[0849] The system according to claim 1, which aims to improve operational efficiency and human resource growth by easily improving performance using information centrally managed on a storage medium.

[0850] "Example 2 of combining an emotion engine"

[0851] (Claim 1)

[0852] Means for collecting operational information related to sales operations,

[0853] A means of centrally controlling the collected information and storing it in an information repository that can be accessed immediately,

[0854] A means of analyzing customer interaction content and generating evaluations using natural language processing,

[0855] A means of providing a real-time response based on the analysis results and the emotion analysis engine,

[0856] A means to automatically optimize work shifts considering working hours and workload,

[0857] A system that includes means for providing educational programs based on each individual's work performance and emotional state.

[0858] (Claim 2)

[0859] The system according to claim 1, which automatically generates individualized educational programs based on work information and emotion analysis.

[0860] (Claim 3)

[0861] The system according to claim 1 aims to improve operational efficiency and foster employee growth by enabling easy improvement of capabilities using information centrally managed on an information storage system.

[0862] "Application example 2 when combining with an emotional engine"

[0863] (Claim 1)

[0864] The status of members involved in sales operations and means of collecting sales data,

[0865] A means of centrally managing the collected information and storing it in a storage device that can be accessed in real time,

[0866] A method for analyzing customer service content and generating evaluations using natural language processing technology,

[0867] A means of providing immediate feedback to members based on the analysis results,

[0868] A means for analyzing the state of customers and members using emotional data from devices worn by members,

[0869] A method to automatically optimize work schedules considering working hours and workload,

[0870] A system that includes a means of proposing training plans based on individual performance evaluations.

[0871] (Claim 2)

[0872] The system according to claim 1, which automatically generates individual training plans based on the results of analyzing business data and emotional information.

[0873] (Claim 3)

[0874] The system according to claim 1, which aims to easily improve performance using centrally managed information in order to improve operational efficiency and foster the growth of its members. [Explanation of Symbols]

[0875] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting behavioral information on human resources in sales operations, A means of centrally managing the collected information and storing it in a storage medium that allows for immediate retrieval, A means of interpreting the content of the interaction and creating an evaluation using language data processing, A means of providing immediate reverse feed based on the interpretation result, A means to automatically optimize work schedules considering work time and workload, A means of proposing an educational plan based on individual behavioral outcomes, A means of instantly displaying customer information and product information when human resources conduct sales activities using a visual assistance device, A system that includes means of analyzing customer interactions through speech recognition and language analysis, and providing information based on the results.

2. The system according to claim 1, comprising means for automatically creating individualized educational plans based on the interpretation results of behavioral information.

3. The system according to claim 1, which aims to improve operational efficiency and human resource growth by easily improving performance using information centrally managed on a storage medium.

Images