system

Abstract

We provide the system. [Solution] A means of entering candidate information, A means of analyzing the entered candidate information to identify the candidate's skill set and career path, A means for generating customized interview questions based on analyzed information, A method for generating a follow-up email to send after the interview, A means of collecting feedback from interviewers and updating the interviewer training program, A system that includes this.

Description

Technical Field

[0004] , , , ,

[0005] , , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In corporate recruitment, due to the low quality of communication between interviewers and applicants, there are issues such as withdrawal from the offer and a decrease in recruitment efficiency. Especially in the IT industry, which requires highly skilled personnel, the variation in interviewers' skills and experience is a major problem. As a result, a decrease in the offer acceptance rate and an increase in recruitment costs are hindering the growth of companies. To solve these problems, there is a need for a method to automatically generate questions and messages for individual candidates and improve the efficiency of interviewer training.

Means for Solving the Problems

[0005] This invention provides a system that includes means for inputting candidate information, means for analyzing the input candidate information to identify the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, means for generating follow-up emails to be sent after the interview, and means for collecting interviewer feedback and updating the interviewer training program. This allows interviewers to ask optimal questions tailored to each candidate, thereby improving the quality of the interview. Furthermore, training based on feedback can improve the interviewers' skills and increase the overall offer acceptance rate.

[0006] "Candidate information" refers to all data about applicants that are evaluated during the hiring process, including resumes, work histories, skill sets, and career paths.

[0007] "Means of input" refers to functions for importing candidate information into the system, and includes components for uploading files and inputting data into a database.

[0008] "Means of analysis" refers to the techniques used to analyze the input candidate information, specifically the process of identifying skill sets and career paths using natural language processing (NLP) and other data analysis techniques.

[0009] "Customized interview questions" refer to individualized questions that are generated based on a candidate's specific skill set and career path.

[0010] A "follow-up email" refers to an email sent to a candidate after an interview, containing confirmation and thank-you messages.

[0011] "Feedback" refers to the evaluations and opinions that interviewers input into a system after an interview, regarding the candidate's reactions and their own performance.

[0012] A "training program" refers to the instruction and education provided to improve interviewers' interviewing skills, and includes content that is updated based on past interview feedback.

[0013] "Generating methods" refer to the process of automatically creating new questions, messages, and follow-up emails using specific algorithms or models. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention relates to a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. This system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs. Specific embodiments for carrying out this invention are described below.

[0036] 1. System Overview

[0037] This system is equipped with programs to perform the following functions:

[0038] Method for entering candidate information

[0039] Means for analyzing the information of entered candidates

[0040] A means of generating customized interview questions based on analysis results.

[0041] A method for generating follow-up emails after an interview.

[0042] A means of collecting feedback from interviewers and updating the training program based on that feedback.

[0043] 2. Data Entry and Analysis

[0044] Users (interviewers or HR personnel) upload candidates' resumes and work histories to the system. File uploads are done via a web interface or a dedicated desktop application. The terminal temporarily stores these files in local storage and then transfers them to the server.

[0045] The server analyzes the received files using natural language processing techniques. Specifically, it uses OCR technology to extract text and identify important skill sets and career paths. The analyzed data is stored in a database.

[0046] 3. Generating Question and Follow-up Emails

[0047] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. This allows for the automatic creation of questions tailored to the candidate's specific skill set and career path. The generated list of questions is then provided to the interviewer via their terminal.

[0048] After the interview, the server generates a follow-up email, which is also sent to the interviewer via the terminal for confirmation. The follow-up email includes a thank-you message and notification of the next steps.

[0049] 4. Feedback and Training Program

[0050] After the interview is complete, the user (interviewer) enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations.

[0051] The server analyzes the interviewer's performance based on the collected feedback and updates the training program for the next interview. This training program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[0052] Specific example

[0053] For example, if a candidate's resume is uploaded and that candidate has "Python development experience," "data analysis skills," and "project management experience," the server will automatically generate interview questions like the following.

[0054] "What was the most challenging aspect of your project using Python?"

[0055] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0056] "What lessons have you learned from your project management experience?"

[0057] Furthermore, the follow-up email sent after the interview will be generated with content similar to the following.

[0058] "Thank you for participating in the interview. We will contact you regarding the next steps shortly."

[0059] In this way, the system can improve the quality of interviews and increase the offer acceptance rate.

[0060] The following describes the processing flow.

[0061] Step 1:

[0062] Users can upload candidates' resumes and work histories by dragging and dropping them into the system's upload form, or by selecting them from a file selection dialog.

[0063] Step 2:

[0064] The device temporarily saves the uploaded file to local storage and displays a progress bar to the user that says, "Sending data to the server."

[0065] Step 3:

[0066] The server receives the file sent from the terminal and stores it in the initial database. After that, it starts processing to convert the file contents into text format.

[0067] Step 4:

[0068] The server uses OCR (Optical Character Recognition) technology and libraries that extract text according to the file format to obtain the text content of the resume.

[0069] Step 5:

[0070] The server uses a natural language processing (NLP) model to analyze and extract important information such as skill sets, years of experience, educational background, and work history from the extracted text. The analysis results are stored in a database.

[0071] Step 6:

[0072] The terminal visualizes the analyzed data and displays a list of skill keywords and the candidate's career path on the screen. This allows interviewers to see which skills and experiences they should focus on evaluating.

[0073] Step 7:

[0074] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. The questions are generated to correspond to the candidate's specific skill set and career path.

[0075] Step 8:

[0076] The terminal displays a generated list of questions to the interviewer. The interviewer can select appropriate questions from the displayed list and edit them as needed.

[0077] Step 9:

[0078] The user conducts interviews with candidates using questions provided by the system. Data and observations obtained during the interview process are then entered into the system.

[0079] Step 10:

[0080] The server automatically generates follow-up emails to send after the interview using an AI model. These follow-up emails include expressions of gratitude and notifications for the next steps.

[0081] Step 11:

[0082] The device displays the contents of the follow-up email to the interviewer, informing them that they can review and revise it.

[0083] Step 12:

[0084] After the interview, the user (interviewer) uses a feedback form to evaluate the quality of the interview and the candidate's responses. The evaluation results are entered into the system.

[0085] Step 13:

[0086] The server receives feedback data and analyzes the interviewer's performance. Based on this analysis, it updates the training program for the next interview.

[0087] Step 14:

[0088] The device displays updated training programs and feedback results to the interviewer, providing information to help them prepare for the next interview.

[0089] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[0090] (Example 1)

[0091] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] In a company's recruitment process, it is crucial to streamline communication between interviewers and applicants and increase the offer acceptance rate. However, traditional manual interview preparation and follow-up email creation are time-consuming and labor-intensive, and suffer from inconsistencies in consistency and quality. Furthermore, insufficient interviewer training can lead to low interview quality. A system is needed to address these challenges and realize a more effective and efficient recruitment process.

[0093] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0094] In this invention, the server includes means for the user to input candidate information, means for the terminal to temporarily store the entered candidate information in local storage and transfer it to the server, means for the server to analyze the received candidate information and identify the candidate's skill set and career path, means for generating customized interview questions using a generative AI model based on the analyzed information, means for generating follow-up emails using the generative AI model after the interview is completed, and means for collecting interviewer feedback from the user and for the server to update the training program based on that feedback. This enables the system to streamline interview preparation, automate follow-up emails, and optimize interviewer training.

[0095] A "user" is a person or person who uses the system to input candidate information or interviewer feedback.

[0096] A "terminal" refers to a device such as a computer or tablet used by a user to temporarily store candidate information and transfer it to a server.

[0097] A "server" is the central hub of the entire system, a computer system that performs tasks such as analyzing candidate information, generating customized interview questions, creating follow-up emails, and updating training programs.

[0098] "Candidate information" refers to the content of resumes and work experience summaries obtained during the recruitment process.

[0099] "Local storage" refers to the area on a device where data is temporarily stored.

[0100] "Transfer" refers to the process of sending candidate information from a terminal to a server.

[0101] "Analysis" refers to the process of using natural language processing technology to analyze the received candidate information and identify their skill set and career path.

[0102] A "generative AI model" is an artificial intelligence model used to automatically generate customized interview questions and follow-up emails.

[0103] "Customized interview questions" are specific questions created by a generative AI model based on analyzed candidate information.

[0104] A "follow-up email" is an email sent to a candidate after an interview to express gratitude and provide information about the next steps.

[0105] "Feedback" refers to the opinions and comments that interviewers enter after an interview regarding the candidate's responses, the appropriateness of the questions, and other observations.

[0106] A "training program" is educational content that the server updates based on feedback to improve the interviewer's performance.

[0107] Modes for carrying out the invention

[0108] This invention is a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. The system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs.

[0109] Hardware and software to be used

[0110] hardware

[0111] Server: This is the central hub of the entire system, performing data analysis, generating questions using generative AI models, creating follow-up emails, and updating training programs.

[0112] Device: A device such as a computer or tablet used by the user (interviewer or HR representative).

[0113] software

[0114] Web Interface: Users upload candidate information via a web page using HTML / CSS / JavaScript.

[0115] OCR technology: Use Tesseract OCR to extract text from candidates' resumes and work histories.

[0116] Natural Language Processing (NLP) Libraries: Use Python-based NLP libraries (e.g., spaCy or NLTK) to analyze text and identify skill sets and career paths.

[0117] Generative AI Model: Uses OpenAI's GPT-3 and other technologies to generate customized interview questions and follow-up emails.

[0118] Database: We use MySQL or PostgreSQL to store analysis results and feedback data.

[0119] Specific operation of the system

[0120] 1. Data entry

[0121] Users upload resumes and work histories of interview candidates to the system. Uploads can be made via a web interface or a dedicated desktop application. For example, a user can select a file from their PC and press the upload button, and the file will be sent to the server.

[0122] 2. Data Analysis

[0123] The terminal temporarily saves uploaded files to local storage and then transfers them to the server. The server uses Tesseract OCR to extract text from the files and analyzes the text using Python's NLP library to identify important skill sets and career paths. The analyzed data is stored in a database and used for subsequent processing.

[0124] 3. Generating interview questions

[0125] Based on the analysis results, the server uses a generation AI model to generate customized interview questions. As a specific example, enter the following prompt text:

[0126] Skills extracted from the candidate's resume:

[0127] Python development experience

[0128] Data analysis skills

[0129] Project management experience

[0130] Based on the above skill set, generate questions that interviewers should ask candidates.

[0131] Based on this prompt, the generative AI model generates the following question.

[0132] 1. "What was the most challenging aspect of your project using Python?"

[0133] 2. "Please tell us about the specific tools and techniques you used when performing data analysis."

[0134] 3. "What lessons have you learned from your project management experience?"

[0135] 4. Conducting interviews and generating follow-up emails

[0136] The user conducts interviews with candidates using a list of questions provided through their device. After the interview, the server generates a follow-up email using a generative AI model. The follow-up email includes an expression of gratitude and a notification about the next steps. For example, it might contain the following:

[0137] Subject: Thank you for participating in the interview.

[0138] Main text:

[0139] [Candidate Name],

[0140] Thank you very much for participating in our interview. We will contact you regarding the next steps shortly. If you have any questions, please do not hesitate to contact us.

[0141] Thank you for your cooperation.

[0142] [Company Name] Recruitment Team

[0143] 5. Gathering feedback and updating the training program

[0144] After an interview, the user (interviewer) enters feedback through the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations. The server analyzes the collected feedback and evaluates the interviewer's performance. A training program for the next interview is automatically updated.

[0145] In this way, this system can improve the quality of interviews and increase the offer acceptance rate.

[0146] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0147] Step 1: User uploads candidate information.

[0148] Users upload resumes and work histories of interview candidates to the system. Input consists of candidate document files (PDF or Word files), primarily using a web interface or a dedicated desktop application. Specifically, users click a file selection button, select a file from their PC, and click the upload button. This action sends the file to the server.

[0149] Step 2: Saving files locally on the terminal and transferring them to the server.

[0150] The terminal temporarily saves the user-uploaded file to local storage. The file is then transferred to the specified server endpoint using an HTTP POST request. The input is the file uploaded by the user, and the output is the file transferred to the server.

[0151] Step 3: Server analyzes candidate information

[0152] The server analyzes the received file. First, it extracts text from the file using OCR technology (e.g., Tesseract OCR). This text extraction result is then analyzed using a Python NLP library (e.g., spaCy) to identify important skill sets and career paths. The input is the file transferred to the server, and the output is the analyzed information on the candidate's skill sets and career paths. Specifically, the server performs OCR processing and converts the resulting text data into structured data using natural language processing techniques.

[0153] Step 4: Server generates interview questions

[0154] The server generates customized interview questions using a generative AI model (e.g., OpenAI's GPT-3) based on the analysis results. The prompt text is input to the generative AI model, and the generated question list is obtained. The input is the analyzed candidate information and prompt text, and the output is a list of generated interview questions. The following prompt text is used as a concrete example.

[0155] Skills extracted from the candidate's resume:

[0156] Python development experience

[0157] Data analysis skills

[0158] Project management experience

[0159] Based on the above skill set, generate questions that interviewers should ask candidates.

[0160] The generative AI model generates a list of questions based on this.

[0161] Step 5: Conducting interviews with users

[0162] The user conducts interviews with candidates using a list of interview questions provided via the terminal. In this step, the user conducts actual interviews based on the generated question list and evaluates the candidates' responses. The input is the generated interview question list, and the output is the interview results.

[0163] Step 6: Server generates follow-up email

[0164] After the interview ends, the server uses a generative AI model to generate a follow-up email. The follow-up email includes an expression of gratitude and notification of the next steps. The input is the interview completion information, and the output is the generated follow-up email. As a specific example, the following follow-up email is generated:

[0165] Subject: Thank you for participating in the interview.

[0166] Main text:

[0167] [Candidate Name],

[0168] Thank you very much for participating in our interview. We will contact you regarding the next steps shortly. If you have any questions, please do not hesitate to contact us.

[0169] Thank you for your cooperation.

[0170] [Company Name] Recruitment Team

[0171] Step 7: User enters interview feedback.

[0172] After the interview, the user enters feedback through the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations. The input is interview feedback information, and the output is feedback data stored on the server.

[0173] Step 8: Server updates the training program

[0174] The server analyzes the collected feedback, evaluates the interviewer's performance, and updates the training program for the next interview. The input is feedback data, and the output is the updated training program. The server integrates this training program with other feedback data and interview data to provide optimal improvement suggestions.

[0175] (Application Example 1)

[0176] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0177] The goal is to improve the quality of customer service in physical stores and increase customer satisfaction. In particular, there is a need for a system that efficiently generates customized customer service questions based on customer interests and purchase history, automatically generates follow-up emails after customer service, and collects staff feedback and updates training programs.

[0178] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0179] In this invention, the server includes means for inputting customer information, means for analyzing the information to identify customer interests and purchase history, means for generating customized customer service questions, means for generating follow-up emails after the customer service is completed, and means for collecting staff feedback and updating training programs. This makes it possible to provide high-quality customer service tailored to the individual needs of customers and improve customer satisfaction.

[0180] A "customer" is a person who visits a commercial facility or service provider, and is the provider of information that identifies their interests and purchasing history.

[0181] "Information analysis" refers to the act of processing input data and extracting meaning and patterns from it, and in this invention, it is used to identify customer interests and purchase history.

[0182] "Customized customer service questions" are questions that are dynamically generated based on each customer's interests and purchase history, and are designed to provide individualized service.

[0183] A "follow-up email" is an email sent to a customer after the service has concluded, containing a message of gratitude and notification of the next steps.

[0184] "Staff" refers to employees who are engaged in providing services to customers in a physical store, and are the subjects of feedback and training programs.

[0185] "Feedback" refers to the opinions and evaluations that staff members have regarding customer service, and the training program is updated based on this feedback.

[0186] A "training program" refers to educational content aimed at improving staff customer service skills, and it is optimized based on feedback.

[0187] "Natural language processing technology" refers to a set of technologies used in the fields of computer science and artificial intelligence to understand and generate natural language, and is used in information analysis.

[0188] "Uploading data via the internet" refers to the act of transmitting customer information or other data to a system using an online network.

[0189] Modes for carrying out the invention

[0190] 1. System Program

[0191] Explain the program's processing in natural language.

[0192] This system is designed to provide customized customer service in physical stores. The specific processing flow is described below.

[0193] Customer information input and analysis

[0194] Users (staff) input basic customer information and purchase history into the system using devices such as smartphones and smart glasses. The devices transmit this information to the server via the internet. The server analyzes the received data using natural language processing techniques to identify customer interests and purchase history. This analysis uses natural language processing libraries such as Spacy and generative AI models such as GPT-3.

[0195] Generating customized customer service questions

[0196] The server generates prompts based on the analysis results, which use a generative AI model to generate customized customer service questions. The generated questions are then provided to the staff via the terminal. For example, for a customer with "Python development experience," "data analysis skills," and "project management experience," the following questions might be generated:

[0197] "What was the most challenging aspect of your project using Python?"

[0198] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0199] "What lessons have you learned from your project management experience?"

[0200] Generating follow-up emails

[0201] After the customer service session ends, the server automatically generates a follow-up email using a generation AI model. This email includes a message of gratitude and encourages the customer to visit again. The generated email is reviewed by staff via their terminals and then sent to the customer.

[0202] Example: "Thank you for visiting our store today. We look forward to your next visit."

[0203] Feedback collection and training program updates

[0204] Users (staff) input feedback about customer service using a terminal. This feedback includes customer reactions and the appropriateness of questions asked. The server stores this feedback information in a database and uses it to update staff training programs. This training program is optimized based on the collected feedback and reflected in future customer service interactions.

[0205] Specific example

[0206] Let's say a customer comes into the store and has "Python development experience," "data analysis skills," and "project management experience." The user (staff) enters the customer's basic information on their smartphone. The server analyzes this information and uses a generative AI model to generate customized questions like the following.

[0207] "What was the most challenging aspect of your project using Python?"

[0208] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0209] "What lessons have you learned from your project management experience?"

[0210] After serving a customer, the server automatically generates a follow-up email using a generative AI model, such as the following:

[0211] "Thank you for visiting our store today. We look forward to your next visit."

[0212] Example of a prompt

[0213] Prompt: "Generate customer service questions for a person with experience in Python development, data analysis, and project management."

[0214] In this way, this system can improve the quality of customer service in physical stores and increase customer satisfaction.

[0215] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0216] Step 1:

[0217] Users input basic customer information and purchase history using devices such as smartphones or smart glasses. Specifically, they enter data such as the customer's name, age, and past purchases into input fields. This data is transmitted from the device to the server via the internet. The input data consists of the customer's basic information and purchase history.

[0218] Step 2:

[0219] The server receives basic customer information and purchase history transmitted from the terminal. Next, natural language processing techniques (such as Spacy) are used to analyze this data and identify customer interests and purchasing patterns. As a result of the data analysis, the customer's areas of interest and purchasing tendencies are identified. The analyzed information includes the customer's interests and purchase history.

[0220] Step 3:

[0221] The server generates prompts based on the analysis results. For example, if "Python development experience," "data analysis skills," and "project management experience" are identified, the prompt might be "Generate customer service questions for a person with experience in Python development, data analysis, and project management." The generated prompts serve as input data for generating customized customer service questions.

[0222] Step 4:

[0223] The server uses a generative AI model (such as GPT-3) to take prompt text as input and generate customized customer service questions. Specific questions include, "What was the most challenging aspect of your project using Python?" The generated list of questions is sent to the terminal as customized customer service questions and provided to the staff. The prompt text is used as input data, and the generated customer service questions are used as output data.

[0224] Step 5:

[0225] The user (staff) interacts with the customer based on customized questions displayed on the terminal. Information obtained during the interaction can also be added to the input fields as needed. The entered data constitutes additional customer information.

[0226] Step 6:

[0227] After the customer service is complete, the server automatically generates a follow-up email using a generation AI model. For example, a follow-up email with content such as, "Thank you for visiting our store today. We look forward to your next visit," is generated. The generated follow-up email is sent to the customer's terminal as a follow-up message. The customer service results are used as input data, and the generated follow-up email is used as output data.

[0228] Step 7:

[0229] Users (staff) input feedback on customer service using a terminal. This feedback includes customer reactions and the appropriateness of questions asked. The input feedback data is sent to a server via the internet. The feedback serves as an evaluation of the staff's customer service and identifies areas for improvement.

[0230] Step 8:

[0231] The server receives feedback sent from the terminal and stores it in a database. It then updates the training program based on the collected feedback. For example, if feedback indicates that "the questions about Python were too difficult," "adjusting the difficulty of questions" will be added to the next training program. The updated training program is saved as instructional items for improving staff skills. The input data is the collected feedback, and the output data is the updated training program.

[0232] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0233] This invention relates to a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. This system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs. Furthermore, by incorporating an emotion engine, it has the function of recognizing and analyzing the emotions of interviewers and candidates in real time. Specific embodiments for carrying out this invention are described below.

[0234] 1. System Overview

[0235] This system is equipped with programs to perform the following functions:

[0236] Method for entering candidate information

[0237] Means for analyzing the information of entered candidates

[0238] A means of generating customized interview questions based on analysis results.

[0239] A method for generating follow-up emails after an interview.

[0240] A means of collecting feedback from interviewers and updating the training program based on that feedback.

[0241] A means of combining an emotion engine that recognizes the emotions of interviewers and candidates.

[0242] A means of analyzing emotional data and providing real-time feedback to interviewers based on that data.

[0243] A means of customizing training programs based on emotional data

[0244] 2. Data Entry and Analysis

[0245] Users (interviewers or HR personnel) upload candidates' resumes and work histories to the system. File uploads are done via a web interface or a dedicated desktop application. The terminal temporarily stores these files in local storage and then transfers them to the server.

[0246] The server analyzes the received files using natural language processing techniques. Specifically, it uses OCR technology to extract text and identify important skill sets and career paths. The analyzed data is stored in a database.

[0247] 3. Generating Question and Follow-up Emails

[0248] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. This allows for the automatic creation of questions tailored to the candidate's specific skill set and career path. The generated list of questions is then provided to the interviewer via their terminal.

[0249] After the interview, the server generates a follow-up email, which is also sent to the interviewer via the terminal for confirmation. The follow-up email includes a thank-you message and notification of the next steps.

[0250] 4. Feedback and Training Program

[0251] After the interview is complete, the user (interviewer) enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations.

[0252] The server analyzes the interviewer's performance based on the collected feedback and updates the training program for the next interview. This training program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[0253] 5. Integration of the Emotional Engine

[0254] The server uses an emotion engine that recognizes emotions from the voice and facial expressions of both the interviewer and the candidate during the interview. The emotion engine collects emotional data from voice tone, changes in facial expression, word choice, etc., and analyzes it in real time.

[0255] This emotional data is fed back to the interviewer in real time via the device, supporting smoother communication during the interview. Furthermore, the emotional data collected after the interview is used to customize the interviewer's training program.

[0256] Specific example

[0257] For example, if a candidate's resume is uploaded and that candidate has "Python development experience," "data analysis skills," and "project management experience," the server will automatically generate interview questions like the following.

[0258] "What was the most challenging aspect of your project using Python?"

[0259] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0260] "What lessons have you learned from your project management experience?"

[0261] During the interview, the server uses an emotion engine to analyze the emotions of both the interviewer and the candidate in real time, sending feedback to the terminal such as, "The candidate is nervous about this question." This allows the interviewer to adjust how they ask questions.

[0262] The follow-up email sent after the interview will contain content similar to the following.

[0263] "Thank you for participating in the interview. We will contact you regarding the next steps shortly."

[0264] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[0265] The following describes the processing flow.

[0266] Step 1:

[0267] Users upload candidates' resumes and work histories to the system. Users can drag and drop files or select them from a file selection dialog using a web interface or a dedicated desktop application.

[0268] Step 2:

[0269] The device temporarily saves the uploaded file to local storage and then transfers the file to the server. During this process, an upload progress bar is displayed to inform the user of the progress.

[0270] Step 3:

[0271] The server receives the file sent from the terminal and stores it in the initial database. Next, it starts the process of converting the file to a text format according to its format.

[0272] Step 4:

[0273] The server extracts the text content from the resume using OCR (Optical Character Recognition) technology and text extraction libraries tailored to the file format.

[0274] Step 5:

[0275] The server uses natural language processing (NLP) technology to analyze and extract important information such as skill sets, years of experience, education background, work history, etc. from the extracted text. The analysis results are stored in a database.

[0276] Step 6:

[0277] The terminal visualizes the analyzed data and displays on the screen a keyword list of skills and the career paths of candidates. This enables the interviewer to confirm which skills and experiences should be evaluated with emphasis.

[0278] Step 7:

[0279] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. The questions are generated corresponding to the specific skill sets and career paths of the candidates.

[0280] Step 8:

[0281] The terminal displays the generated question list to the interviewer. The interviewer can select appropriate questions from the displayed question list and edit them if necessary.

[0282] Step 9:

[0283] The server activates an emotion engine for recognizing emotions from the voices and expressions of the interviewer and the candidate during the interview. The emotion engine collects emotion data from voice tones, changes in expressions, word choices, etc. and analyzes them in real time.

[0284] Step 10:

[0285] Based on the emotion data obtained by the emotion engine, the terminal provides real-time feedback to the interviewer. This feedback includes specific advice such as "The candidate is nervous" or "The interviewer should be more relaxed".

[0286] Step 11:

[0287] The user (interviewer) conducts an interview with the candidate using the questions provided by the system. The data and observations obtained during the interview are input into the system.

[0288] Step 12:

[0289] The server automatically generates a follow-up email to be sent after the interview using the generation AI model. The follow-up email includes an expression of gratitude and a notification of the next step.

[0290] Step 13:

[0291] The terminal displays the content of the follow-up email to the interviewer and informs that it can be confirmed and corrected.

[0292] Step 14:

[0293] The user (interviewer) uses a feedback form after the interview to evaluate the quality of the interview and the candidate's reaction. The evaluation results are input into the system.

[0294] Step 15:

[0295] The server receives the feedback data and conducts a performance analysis of the interviewer. Based on this analysis result, the training program for the next interview is updated.

[0296] Step 16: [[ID=四十一]]

[0297] The server analyzes the sentiment data collected by the sentiment engine and customizes the interviewer's training program based on it. As a result, the interviewer can receive specific advice to improve their sentiment recognition skills and communication skills.

[0298] Step 17:

[0299] The device displays updated training programs and feedback results to the interviewer, providing information to help them prepare for the next interview.

[0300] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[0301] (Example 2)

[0302] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0303] Traditional recruitment processes lacked the ability to generate appropriate questions and provide real-time feedback to facilitate smooth communication between interviewers and applicants. Furthermore, sending follow-up emails after interviews and updating interviewer training programs were often done manually, which was inefficient. Additionally, the lack of automated recognition and feedback on emotions during interviews made it difficult to improve the quality of communication.

[0304] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0305] In this invention, the server includes means for inputting candidate information, means for analyzing the input candidate information and identifying the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, means for generating follow-up emails to be sent after the interview, means for collecting interviewer feedback and updating the interviewer's training program, means for combining an emotion engine that recognizes the emotions of the interviewer and candidate during the interview, means for analyzing the emotion data and providing real-time feedback to the interviewer based on it, and means for customizing the training program based on the emotion data. This makes it possible to improve the quality of communication between interviewers and applicants and to increase the efficiency and effectiveness of the entire recruitment process.

[0306] The "means for inputting candidate information" refers to the interface or device for an interviewer or HR staff to input information such as a candidate's resume or work history into the system.

[0307] The "means for analyzing the input candidate information and identifying the candidate's skill set and career path" refers to the technology and process for the system to analyze the information provided by the candidate using natural language processing technology and extract important skills and work history.

[0308] The "means for generating customized interview questions based on the analyzed information" refers to the technology and process for automatically creating individually suitable interview questions using a generation AI model based on the analysis results.

[0309] The "means for generating a follow-up email to be sent after the interview" refers to the technology and process for automatically generating and sending an email including expressions of gratitude and notifications regarding the next steps after the interview.

[0310] The "means for collecting interviewer feedback and updating the interviewer training program" refers to the technology and process for collecting feedback information from the interviewer after the interview and updating the training program for continuously improving the interviewer's skills and knowledge based on it.

[0311] The "means for combining an emotion engine that recognizes the emotions of the interviewer and the candidate during the interview" refers to the function implementing emotion recognition technology used to analyze emotions from voice, expressions, etc. during the interview.

[0312] The "means for analyzing the emotion data and providing real-time feedback to the interviewer based on it" refers to the technology and process for analyzing the collected emotion data, notifying the analysis results in real time, and assisting the interviewer's response.

[0313] "Methods for customizing training programs based on emotional data" refers to techniques and processes for analyzing emotional data collected during interviews and using the results to customize and improve the interviewer's training program.

[0314] This invention is a system designed to improve communication between interviewers and applicants in a company's recruitment process, while simultaneously increasing the offer acceptance rate. This system functions by combining multiple software and hardware elements. Specific embodiments for carrying out this invention are described below.

[0315] 1. Data entry and temporary storage

[0316] Users upload candidate resumes and CVs to the system using a web interface or a dedicated desktop application. This involves selecting files and clicking an upload button. The terminal temporarily saves the uploaded files to local storage and then transfers them to the server.

[0317] The hardware used includes personal computers, tablets, and smartphones used by the user. The software includes web browsers and dedicated desktop applications.

[0318] 2. Transferring and analyzing data to the server

[0319] The device transfers saved files to the server using a secure communication protocol (e.g., HTTPS). The server extracts text data from the uploaded files using OCR (Optical Character Recognition) technology. This text data is then analyzed using natural language processing (NLP) technology to identify important skill sets and career paths.

[0320] Specific software includes libraries for using OCR technology (e.g., Tesseract) and frameworks using NLP technology (e.g., NLTK, spaCy). The analysis results are stored in a database.

[0321] 3. Generating interview questions and follow-up emails

[0322] The server uses a generative AI model (e.g., OpenAI's GPT, BERT) to generate customized interview questions based on the analysis results. The generated list of questions is stored in a database, and a notification is sent to the terminal. The terminal displays the received notification to the interviewer, who can then view the list of questions via a web interface or application.

[0323] After the interview, the server automatically generates a follow-up email and requests confirmation from the interviewer via the terminal. This email contains an expression of gratitude and information about the next steps.

[0324] For example, if a candidate has "Python development experience," "data analysis skills," and "project management experience," the server will generate interview questions like the following:

[0325] "What was the most challenging aspect of your project using Python?"

[0326] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0327] "What lessons have you learned from your project management experience?"

[0328] 4. Gathering feedback and updating the training program

[0329] After the interview, the user enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and an overall evaluation.

[0330] The server analyzes the collected feedback and stores it in a database as interviewer performance data. It then updates the training program for the next interview. This program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[0331] 5. Integration of the emotion engine and real-time feedback

[0332] The server uses an emotion engine to collect voice and facial expression data from interviewers and candidates in real time during interviews. The emotion engine estimates and analyzes emotional states based on voice tone, changes in facial expressions, and word choice.

[0333] The server sends the analysis results to the terminal in real time and provides feedback to the interviewer. Specific feedback may include comments such as, "The candidate appears nervous about this question."

[0334] Example of a prompt

[0335] You can use the following prompt statements to input data into the generative AI model and obtain the appropriate output.

[0336] "Analyze the next candidate's resume and generate customized interview questions."

[0337] "Please update the training program based on the interview feedback."

[0338] "Analyze the emotional data during the interview and provide real-time feedback."

[0339] In this way, the system can automate a series of processes, aiming to improve the quality of interviews and the offer acceptance rate.

[0340] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0341] Step 1:

[0342] Users upload candidate resumes and CVs using a web interface or a dedicated desktop application. The user selects the file and clicks the upload button. The input is the candidate's resume or CV, and the output is the uploaded file. Specifically, a file selection dialog opens, the user selects the file, and clicks the "Upload" button.

[0343] Step 2:

[0344] The terminal temporarily saves uploaded files to local storage. The input is the uploaded file, and the output is the file saved in local storage. Specifically, the system retrieves basic file information (filename, size, upload time, etc.) and saves the file to local storage.

[0345] Step 3:

[0346] The terminal transfers saved files to the server using a secure communication protocol (e.g., HTTPS). The input is a file stored in local storage, and the output is a file transferred to the server. Specifically, the terminal sends a secure HTTP request to the server and transfers the file data.

[0347] Step 4:

[0348] The server analyzes the received file using OCR technology. The input is the file uploaded to the server, and the output is the analyzed text data. Specifically, the server uses the Tesseract library to extract the text data from the file and saves it to a database.

[0349] Step 5:

[0350] The server analyzes text data extracted using natural language processing technology. The input is text data extracted by OCR, and the output is the identification of candidates' skill sets and career paths. Specifically, the server uses NLTK and spaCy to analyze the text data and identify important skill sets and work history information.

[0351] Step 6:

[0352] The server generates customized interview questions using a generative AI model based on the analysis results. The input is the analyzed candidate information, and the output is the generated list of interview questions. Specifically, the server inputs prompt sentences into the generative AI model (e.g., OpenAI's GPT) and generates the list of questions.

[0353] Step 7:

[0354] The server saves the generated question list to a database and sends a notification to the terminal. The input is the generated interview question list, and the output is the notification message. Specifically, the server saves the question list to the database and sends a notification message to the terminal.

[0355] Step 8:

[0356] The terminal displays notifications received from the server to the interviewer. The input is the notification message from the server, and the output is a display of the question list that the interviewer can view. Specifically, the terminal displays the notification message in the user interface, and clicking it displays the details of the question list.

[0357] Step 9:

[0358] The server automatically generates a follow-up email after the interview is completed. The input is the interview completion information, and the output is the follow-up email. Specifically, the server automatically generates the email content based on a follow-up email template and sends it to the terminal.

[0359] Step 10:

[0360] The terminal displays the follow-up email received from the server to the interviewer and requests their confirmation of its contents. The input is the received follow-up email, and the output is the follow-up email confirmed by the interviewer. Specifically, the terminal displays the contents of the follow-up email, allowing the interviewer to make corrections or approve it.

[0361] Step 11:

[0362] The user inputs interview feedback into the system interface. The input is the interviewer's feedback, and the output is feedback data. Specifically, the user enters the feedback into the input field and presses the submit button.

[0363] Step 12:

[0364] The server analyzes the collected feedback and updates the training program. The input is the collected feedback data, and the output is the updated training program. Specifically, the server analyzes the feedback data, generates the training content, and saves it to the database.

[0365] Step 13:

[0366] The server uses an emotion engine to collect and analyze emotional data during the interview. Inputs are voice and facial expression data, and output is the analyzed emotional data. Specifically, the server uses emotion recognition technology to analyze the emotional data in real time and provides it to the interviewer as feedback.

[0367] Step 14:

[0368] The server provides real-time feedback to the interviewer based on emotional data. The input is analyzed emotional data, and the output is a feedback message. Specifically, the server analyzes the emotional data and sends it to the interviewer's terminal so that they can receive visualized feedback.

[0369] Step 15:

[0370] The server customizes the training program based on emotional data. The input is emotional data, and the output is the customized training program. Specifically, the server analyzes the emotional data, optimizes the training content for the interviewer, and stores it in a database.

[0371] (Application Example 2)

[0372] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0373] This invention relates to a system for improving efficiency and accuracy in conventional recruitment processes and the work performance of factory robots. Conventional recruitment processes often rely heavily on the subjective opinions and experience of interviewers, resulting in lower offer acceptance rates. Similarly, in factory robot operations, delays in operator feedback can reduce efficiency and increase the likelihood of errors. This invention aims to solve these problems and improve the quality of interviews and factory work.

[0374] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0375] In this invention, the server includes means for analyzing candidate information and identifying the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, and means equipped with an emotion engine for collecting data in real time and providing feedback to the operator. This enables the provision of objective and customized questions to the interviewer, and improves work efficiency and accuracy by providing real-time feedback based on robotic and human emotion data.

[0376] A "candidate" is an individual who possesses the necessary information and skills to apply for a specific position.

[0377] "Means for analyzing information" refers to a device or program that has the function of identifying specific attributes or patterns based on input data and generating the necessary output based on those attributes or patterns.

[0378] A "skill set" refers to the abilities and experience required to perform a specific job or role.

[0379] A "career path" is a diagram or outline that shows an individual's work history and professional experience, illustrating the future progression of their career.

[0380] "Means for generating customized interview questions" refers to a device or program that automatically creates questions optimized for individual candidates based on analysis results.

[0381] A "follow-up email" is an email sent to a candidate after an interview, containing a thank you and information about the next steps.

[0382] An "interviewer" is an individual responsible for conducting interviews with candidates and evaluating their performance.

[0383] A "training program" is a curriculum of education and training designed to improve the skills and performance of interviewers.

[0384] "Means of collecting data in real time" refers to devices or programs that have the function of immediately acquiring and analyzing data generated on-site.

[0385] An "emotion engine" is a device or program that recognizes and analyzes emotions based on a person's facial expressions, tone of voice, and choice of words.

[0386] An "operator" is an individual responsible for operating machinery or equipment.

[0387] "Means of providing feedback" refers to a device or program that notifies operators or users in real time of suggestions for improvement or precautions based on analysis results.

[0388] 1. System Configuration

[0389] This invention provides a system for improving the work performance of factory robots. The system includes the following components:

[0390] 1. Means of entering candidate information

[0391] 2. A means of analyzing the entered candidate information to identify the candidate's skill set and career path.

[0392] 3. Means for generating customized interview questions based on analyzed information

[0393] 4. A method for generating follow-up emails to send after the interview has concluded.

[0394] 5. Means of collecting interviewer feedback and updating interviewer training programs.

[0395] 6. A system equipped with an emotion engine to collect data in real time and provide feedback to operators.

[0396] 7. A means of analyzing robot and human emotional data and providing real-time feedback to the operator based on the results.

[0397] 2. Program Processing

[0398] The program for realizing this invention performs the following processes:

[0399] Hardware and software usage

[0400] Camera: A video capture device is used to monitor the operation of factory robots. The camera captures the robot's movements as video in real time.

[0401] Emotion Recognition Model: Use an emotion recognition model trained with TensorFlow. Specifically, use the model named emotion_model.h5.

[0402] Face Recognition: Faces are detected using OpenCV's Haar Cascade Classifier. This allows for the analysis of the emotions of the operator and other participants during robot operation.

[0403] Natural Language Processing: Use the TextBlob library to analyze and process text data generated based on the analysis results.

[0404] Sending emails: Use the smtplib library to send notification emails to operators.

[0405] Specific example

[0406] As a concrete example, if a factory robot detects abnormal behavior during operation, the system will operate as follows:

[0407] First, cameras monitor the robot's movements within the factory and acquire video data in real time. The system detects the operator's facial expressions using a Haar cascade classifier and analyzes their emotions using an emotion recognition model (TensorFlow model).

[0408] For example, if an operator expresses emotions such as "surprise" or "anger," the system analyzes that emotion data and determines that a specific anomaly has occurred. In this case, a feedback message like the following is generated:

[0409] "An abnormality has been detected in the robot's operation. Please be careful."

[0410] This feedback message is displayed on the operator's screen in real time and is also notified via email. The smtplib library is used for sending emails.

[0411] Example of a prompt

[0412] Here are some specific examples of prompt statements to input into a generative AI model:

[0413] "Please prepare a detailed report on any issues related to the robot's operation. The report should include the type of anomaly detected, the countermeasures taken, and measures to prevent recurrence."

[0414] These components and processing procedures make it possible to improve the operational performance of factory robots and provide an efficient and safe working environment.

[0415] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0416] Step 1:

[0417] Capture images with a camera

[0418] The terminal acquires real-time video footage of robots and operators operating within the factory from cameras. The input is video data from the cameras, which is processed as a video stream. The output is raw video data.

[0419] Step 2:

[0420] Face detection

[0421] The server detects faces from the acquired video data using OpenCV's Haar cascade classifier. The input is the video data acquired in step 1, which is processed to extract specific areas (faces). The output is face coordinate (x, y, w, h) data.

[0422] Step 3:

[0423] Facial cropping and pre-processing

[0424] The device extracts the face portion from the video data based on the detected face coordinate data and converts it into a format that can be input into the emotion recognition model. The input consists of face coordinate data and raw video data, which are used to extract the face image, resize it, and convert it to grayscale. The output is a pre-processed face image.

[0425] Step 4:

[0426] Recognition of emotions

[0427] The server inputs pre-processed facial images into a TensorFlow emotion recognition model to analyze emotions. The input is pre-processed facial images, and the model predicts emotions. The output is emotion prediction data (labels such as anger, surprise, sadness, etc.).

[0428] Step 5:

[0429] Detection of abnormal operation

[0430] The server analyzes the output data of the emotion recognition model and recognizes abnormal emotions (e.g., anger) as abnormal behavior. The input is emotion prediction data, which is analyzed based on rules for determining whether an abnormality exists. The output is a flag indicating whether an abnormal behavior occurred.

[0431] Step 6:

[0432] Generating feedback messages

[0433] If an abnormal operation is detected, the server uses a generative AI model to generate a feedback message for the operator. The input is an abnormal operation flag and a prompt statement (e.g., "Please prepare a detailed report on the robot's operation issue"), and the AI ​​model generates an appropriate feedback message. The output is the feedback message text.

[0434] Step 7:

[0435] Real-time display of feedback

[0436] The terminal displays the generated feedback message on the operator's display in real time. The input is the feedback message text, and its content is transmitted to the operator via the display device. The output is visual feedback information.

[0437] Step 8:

[0438] Sending notification emails

[0439] The server generates a notification email containing a feedback message and sends it to the operator's email address. The input is the feedback message text and the operator's email address, and the email is sent using the smtplib library. The output is the sent notification email.

[0440] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0441] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0442] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0443] [Second Embodiment]

[0444] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0445] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0446] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0447] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0448] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0449] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0450] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0451] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0452] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0453] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0454] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0455] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0456] This invention relates to a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. This system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs. Specific embodiments for carrying out this invention are described below.

[0457] 1. System Overview

[0458] This system is equipped with programs to perform the following functions:

[0459] Method for entering candidate information

[0460] Means for analyzing the information of entered candidates

[0461] A means of generating customized interview questions based on analysis results.

[0462] A method for generating follow-up emails after an interview.

[0463] A means of collecting feedback from interviewers and updating the training program based on that feedback.

[0464] 2. Data Entry and Analysis

[0465] Users (interviewers or HR personnel) upload candidates' resumes and work histories to the system. File uploads are done via a web interface or a dedicated desktop application. The terminal temporarily stores these files in local storage and then transfers them to the server.

[0466] The server analyzes the received files using natural language processing techniques. Specifically, it uses OCR technology to extract text and identify important skill sets and career paths. The analyzed data is stored in a database.

[0467] 3. Generating Question and Follow-up Emails

[0468] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. This allows for the automatic creation of questions tailored to the candidate's specific skill set and career path. The generated list of questions is then provided to the interviewer via their terminal.

[0469] After the interview, the server generates a follow-up email, which is also sent to the interviewer via the terminal for confirmation. The follow-up email includes a thank-you message and notification of the next steps.

[0470] 4. Feedback and Training Program

[0471] After the interview is complete, the user (interviewer) enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations.

[0472] The server analyzes the interviewer's performance based on the collected feedback and updates the training program for the next interview. This training program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[0473] Specific example

[0474] For example, if a candidate's resume is uploaded and that candidate has "Python development experience," "data analysis skills," and "project management experience," the server will automatically generate interview questions like the following.

[0475] "What was the most challenging aspect of your project using Python?"

[0476] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0477] "What lessons have you learned from your project management experience?"

[0478] Furthermore, the follow-up email sent after the interview will be generated with content similar to the following.

[0479] "Thank you for participating in the interview. We will contact you regarding the next steps shortly."

[0480] In this way, the system can improve the quality of interviews and increase the offer acceptance rate.

[0481] The following describes the processing flow.

[0482] Step 1:

[0483] Users can upload candidates' resumes and work histories by dragging and dropping them into the system's upload form, or by selecting them from a file selection dialog.

[0484] Step 2:

[0485] The device temporarily saves the uploaded file to local storage and displays a progress bar to the user that says, "Sending data to the server."

[0486] Step 3:

[0487] The server receives the file sent from the terminal and stores it in the initial database. After that, it starts processing to convert the file contents into text format.

[0488] Step 4:

[0489] The server uses OCR (Optical Character Recognition) technology and libraries that extract text according to the file format to obtain the text content of the resume.

[0490] Step 5:

[0491] The server uses a natural language processing (NLP) model to analyze and extract important information such as skill sets, years of experience, educational background, and work history from the extracted text. The analysis results are stored in a database.

[0492] Step 6:

[0493] The terminal visualizes the analyzed data and displays a list of skill keywords and the candidate's career path on the screen. This allows interviewers to see which skills and experiences they should focus on evaluating.

[0494] Step 7:

[0495] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. The questions are generated to correspond to the candidate's specific skill set and career path.

[0496] Step 8:

[0497] The terminal displays a generated list of questions to the interviewer. The interviewer can select appropriate questions from the displayed list and edit them as needed.

[0498] Step 9:

[0499] The user conducts interviews with candidates using questions provided by the system. Data and observations obtained during the interview process are then entered into the system.

[0500] Step 10:

[0501] The server automatically generates follow-up emails to send after the interview using an AI model. These follow-up emails include expressions of gratitude and notifications for the next steps.

[0502] Step 11:

[0503] The device displays the contents of the follow-up email to the interviewer, informing them that they can review and revise it.

[0504] Step 12:

[0505] After the interview, the user (interviewer) uses a feedback form to evaluate the quality of the interview and the candidate's responses. The evaluation results are entered into the system.

[0506] Step 13:

[0507] The server receives feedback data and analyzes the interviewer's performance. Based on this analysis, it updates the training program for the next interview.

[0508] Step 14:

[0509] The device displays updated training programs and feedback results to the interviewer, providing information to help them prepare for the next interview.

[0510] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[0511] (Example 1)

[0512] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0513] In a company's recruitment process, it is crucial to streamline communication between interviewers and applicants and increase the offer acceptance rate. However, traditional manual interview preparation and follow-up email creation are time-consuming and labor-intensive, and suffer from inconsistencies in consistency and quality. Furthermore, insufficient interviewer training can lead to low interview quality. A system is needed to address these challenges and realize a more effective and efficient recruitment process.

[0514] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0515] In this invention, the server includes means for the user to input candidate information, means for the terminal to temporarily store the entered candidate information in local storage and transfer it to the server, means for the server to analyze the received candidate information and identify the candidate's skill set and career path, means for generating customized interview questions using a generative AI model based on the analyzed information, means for generating follow-up emails using the generative AI model after the interview is completed, and means for collecting interviewer feedback from the user and for the server to update the training program based on that feedback. This enables the system to streamline interview preparation, automate follow-up emails, and optimize interviewer training.

[0516] A "user" is a person or person who uses the system to input candidate information or interviewer feedback.

[0517] A "terminal" refers to a device such as a computer or tablet used by a user to temporarily store candidate information and transfer it to a server.

[0518] A "server" is the central hub of the entire system, a computer system that performs tasks such as analyzing candidate information, generating customized interview questions, creating follow-up emails, and updating training programs.

[0519] "Candidate information" refers to the content of resumes and work experience summaries obtained during the recruitment process.

[0520] "Local storage" refers to the area on a device where data is temporarily stored.

[0521] "Transfer" refers to the process of sending candidate information from a terminal to a server.

[0522] "Analysis" refers to the process of using natural language processing technology to analyze the received candidate information and identify their skill set and career path.

[0523] A "generative AI model" is an artificial intelligence model used to automatically generate customized interview questions and follow-up emails.

[0524] "Customized interview questions" are specific questions created by a generative AI model based on analyzed candidate information.

[0525] A "follow-up email" is an email sent to a candidate after an interview to express gratitude and provide information about the next steps.

[0526] "Feedback" refers to the opinions and comments that interviewers enter after an interview regarding the candidate's responses, the appropriateness of the questions, and other observations.

[0527] A "training program" is educational content that the server updates based on feedback to improve the interviewer's performance.

[0528] Modes for carrying out the invention

[0529] This invention is a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. The system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs.

[0530] Hardware and software to be used

[0531] hardware

[0532] Server: This is the central hub of the entire system, performing data analysis, generating questions using generative AI models, creating follow-up emails, and updating training programs.

[0533] Device: A device such as a computer or tablet used by the user (interviewer or HR representative).

[0534] software

[0535] Web Interface: Users upload candidate information via a web page using HTML / CSS / JavaScript.

[0536] OCR technology: Use Tesseract OCR to extract text from candidates' resumes and work histories.

[0537] Natural Language Processing (NLP) Libraries: Use Python-based NLP libraries (e.g., spaCy or NLTK) to analyze text and identify skill sets and career paths.

[0538] Generative AI Model: Uses OpenAI's GPT-3 and other technologies to generate customized interview questions and follow-up emails.

[0539] Database: We use MySQL or PostgreSQL to store analysis results and feedback data.

[0540] Specific operation of the system

[0541] 1. Data entry

[0542] Users upload resumes and work histories of interview candidates to the system. Uploads can be made via a web interface or a dedicated desktop application. For example, a user can select a file from their PC and press the upload button, and the file will be sent to the server.

[0543] 2. Data Analysis

[0544] The terminal temporarily saves uploaded files to local storage and then transfers them to the server. The server uses Tesseract OCR to extract text from the files and analyzes the text using Python's NLP library to identify important skill sets and career paths. The analyzed data is stored in a database and used for subsequent processing.

[0545] 3. Generating interview questions

[0546] Based on the analysis results, the server uses a generation AI model to generate customized interview questions. As a specific example, enter the following prompt text:

[0547] Skills extracted from the candidate's resume:

[0548] Python development experience

[0549] Data analysis skills

[0550] Project management experience

[0551] Based on the above skill set, generate questions that interviewers should ask candidates.

[0552] Based on this prompt, the generative AI model generates the following question.

[0553] 1. "What was the most challenging aspect of your project using Python?"

[0554] 2. "Please tell us about the specific tools and techniques you used when performing data analysis."

[0555] 3. "What lessons have you learned from your project management experience?"

[0556] 4. Conducting interviews and generating follow-up emails

[0557] The user conducts interviews with candidates using a list of questions provided through their device. After the interview, the server generates a follow-up email using a generative AI model. The follow-up email includes an expression of gratitude and a notification about the next steps. For example, it might contain the following:

[0558] Subject: Thank you for participating in the interview.

[0559] Main text:

[0560] [Candidate Name],

[0561] Thank you very much for participating in our interview. We will contact you regarding the next steps shortly. If you have any questions, please do not hesitate to contact us.

[0562] Thank you for your cooperation.

[0563] [Company Name] Recruitment Team

[0564] 5. Gathering feedback and updating the training program

[0565] After an interview, the user (interviewer) enters feedback through the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations. The server analyzes the collected feedback and evaluates the interviewer's performance. A training program for the next interview is automatically updated.

[0566] In this way, this system can improve the quality of interviews and increase the offer acceptance rate.

[0567] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0568] Step 1: User uploads candidate information.

[0569] Users upload resumes and work histories of interview candidates to the system. Input consists of candidate document files (PDF or Word files), primarily using a web interface or a dedicated desktop application. Specifically, users click a file selection button, select a file from their PC, and click the upload button. This action sends the file to the server.

[0570] Step 2: Saving files locally on the terminal and transferring them to the server.

[0571] The terminal temporarily saves the user-uploaded file to local storage. The file is then transferred to the specified server endpoint using an HTTP POST request. The input is the file uploaded by the user, and the output is the file transferred to the server.

[0572] Step 3: Server analyzes candidate information

[0573] The server analyzes the received file. First, it extracts text from the file using OCR technology (e.g., Tesseract OCR). This text extraction result is then analyzed using a Python NLP library (e.g., spaCy) to identify important skill sets and career paths. The input is the file transferred to the server, and the output is the analyzed information on the candidate's skill sets and career paths. Specifically, the server performs OCR processing and converts the resulting text data into structured data using natural language processing techniques.

[0574] Step 4: Server generates interview questions

[0575] The server generates customized interview questions using a generative AI model (e.g., OpenAI's GPT-3) based on the analysis results. The prompt text is input to the generative AI model, and the generated question list is obtained. The input is the analyzed candidate information and prompt text, and the output is a list of generated interview questions. The following prompt text is used as a concrete example.

[0576] Skills extracted from the candidate's resume:

[0577] Python development experience

[0578] Data analysis skills

[0579] Project management experience

[0580] Based on the above skill set, generate questions that interviewers should ask candidates.

[0581] The generative AI model generates a list of questions based on this.

[0582] Step 5: Conducting interviews with users

[0583] The user conducts interviews with candidates using a list of interview questions provided via the terminal. In this step, the user conducts actual interviews based on the generated question list and evaluates the candidates' responses. The input is the generated interview question list, and the output is the interview results.

[0584] Step 6: Server generates follow-up email

[0585] After the interview ends, the server uses a generative AI model to generate a follow-up email. The follow-up email includes an expression of gratitude and notification of the next steps. The input is the interview completion information, and the output is the generated follow-up email. As a specific example, the following follow-up email is generated:

[0586] Subject: Thank you for participating in the interview.

[0587] Main text:

[0588] [Candidate Name],

[0589] Thank you very much for participating in our interview. We will contact you regarding the next steps shortly. If you have any questions, please do not hesitate to contact us.

[0590] Thank you for your cooperation.

[0591] [Company Name] Recruitment Team

[0592] Step 7: User enters interview feedback.

[0593] After the interview, the user enters feedback through the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations. The input is interview feedback information, and the output is feedback data stored on the server.

[0594] Step 8: Server updates the training program

[0595] The server analyzes the collected feedback, evaluates the interviewer's performance, and updates the training program for the next interview. The input is feedback data, and the output is the updated training program. The server integrates this training program with other feedback data and interview data to provide optimal improvement suggestions.

[0596] (Application Example 1)

[0597] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0598] The goal is to improve the quality of customer service in physical stores and increase customer satisfaction. In particular, there is a need for a system that efficiently generates customized customer service questions based on customer interests and purchase history, automatically generates follow-up emails after customer service, and collects staff feedback and updates training programs.

[0599] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0600] In this invention, the server includes means for inputting customer information, means for analyzing the information to identify customer interests and purchase history, means for generating customized customer service questions, means for generating follow-up emails after the customer service is completed, and means for collecting staff feedback and updating training programs. This makes it possible to provide high-quality customer service tailored to the individual needs of customers and improve customer satisfaction.

[0601] A "customer" is a person who visits a commercial facility or service provider, and is the provider of information that identifies their interests and purchasing history.

[0602] "Information analysis" refers to the act of processing input data and extracting meaning and patterns from it, and in this invention, it is used to identify customer interests and purchase history.

[0603] "Customized customer service questions" are questions that are dynamically generated based on each customer's interests and purchase history, and are designed to provide individualized service.

[0604] A "follow-up email" is an email sent to a customer after the service has concluded, containing a message of gratitude and notification of the next steps.

[0605] "Staff" refers to employees who are engaged in providing services to customers in a physical store, and are the subjects of feedback and training programs.

[0606] "Feedback" refers to the opinions and evaluations that staff members have regarding customer service, and the training program is updated based on this feedback.

[0607] A "training program" refers to educational content aimed at improving staff customer service skills, and it is optimized based on feedback.

[0608] "Natural language processing technology" refers to a set of technologies used in the fields of computer science and artificial intelligence to understand and generate natural language, and is used in information analysis.

[0609] "Uploading data via the internet" refers to the act of transmitting customer information or other data to a system using an online network.

[0610] Modes for carrying out the invention

[0611] 1. System Program

[0612] Explain the program's processing in natural language.

[0613] This system is designed to provide customized customer service in physical stores. The specific processing flow is described below.

[0614] Customer information input and analysis

[0615] Users (staff) input basic customer information and purchase history into the system using devices such as smartphones and smart glasses. The devices transmit this information to the server via the internet. The server analyzes the received data using natural language processing techniques to identify customer interests and purchase history. This analysis uses natural language processing libraries such as Spacy and generative AI models such as GPT-3.

[0616] Generating customized customer service questions

[0617] The server generates prompts based on the analysis results, which use a generative AI model to generate customized customer service questions. The generated questions are then provided to the staff via the terminal. For example, for a customer with "Python development experience," "data analysis skills," and "project management experience," the following questions might be generated:

[0618] "What was the most challenging aspect of your project using Python?"

[0619] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0620] "What lessons have you learned from your project management experience?"

[0621] Generating follow-up emails

[0622] After the customer service session ends, the server automatically generates a follow-up email using a generation AI model. This email includes a message of gratitude and encourages the customer to visit again. The generated email is reviewed by staff via their terminals and then sent to the customer.

[0623] Example: "Thank you for visiting our store today. We look forward to your next visit."

[0624] Feedback collection and training program updates

[0625] Users (staff) input feedback about customer service using a terminal. This feedback includes customer reactions and the appropriateness of questions asked. The server stores this feedback information in a database and uses it to update staff training programs. This training program is optimized based on the collected feedback and reflected in future customer service interactions.

[0626] Specific example

[0627] Let's say a customer comes into the store and has "Python development experience," "data analysis skills," and "project management experience." The user (staff) enters the customer's basic information on their smartphone. The server analyzes this information and uses a generative AI model to generate customized questions like the following.

[0628] "What was the most challenging aspect of your project using Python?"

[0629] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0630] "What lessons have you learned from your project management experience?"

[0631] After serving a customer, the server automatically generates a follow-up email using a generative AI model, such as the following:

[0632] "Thank you for visiting our store today. We look forward to your next visit."

[0633] Example of a prompt

[0634] Prompt: "Generate customer service questions for a person with experience in Python development, data analysis, and project management."

[0635] In this way, this system can improve the quality of customer service in physical stores and increase customer satisfaction.

[0636] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0637] Step 1:

[0638] Users input basic customer information and purchase history using devices such as smartphones or smart glasses. Specifically, they enter data such as the customer's name, age, and past purchases into input fields. This data is transmitted from the device to the server via the internet. The input data consists of the customer's basic information and purchase history.

[0639] Step 2:

[0640] The server receives basic customer information and purchase history transmitted from the terminal. Next, natural language processing techniques (such as Spacy) are used to analyze this data and identify customer interests and purchasing patterns. As a result of the data analysis, the customer's areas of interest and purchasing tendencies are identified. The analyzed information includes the customer's interests and purchase history.

[0641] Step 3:

[0642] The server generates prompts based on the analysis results. For example, if "Python development experience," "data analysis skills," and "project management experience" are identified, the prompt might be "Generate customer service questions for a person with experience in Python development, data analysis, and project management." The generated prompts serve as input data for generating customized customer service questions.

[0643] Step 4:

[0644] The server uses a generative AI model (such as GPT-3) to take prompt text as input and generate customized customer service questions. Specific questions include, "What was the most challenging aspect of your project using Python?" The generated list of questions is sent to the terminal as customized customer service questions and provided to the staff. The prompt text is used as input data, and the generated customer service questions are used as output data.

[0645] Step 5:

[0646] The user (staff) interacts with the customer based on customized questions displayed on the terminal. Information obtained during the interaction can also be added to the input fields as needed. The entered data constitutes additional customer information.

[0647] Step 6:

[0648] After the customer service is complete, the server automatically generates a follow-up email using a generation AI model. For example, a follow-up email with content such as, "Thank you for visiting our store today. We look forward to your next visit," is generated. The generated follow-up email is sent to the customer's terminal as a follow-up message. The customer service results are used as input data, and the generated follow-up email is used as output data.

[0649] Step 7:

[0650] Users (staff) input feedback on customer service using a terminal. This feedback includes customer reactions and the appropriateness of questions asked. The input feedback data is sent to a server via the internet. The feedback serves as an evaluation of the staff's customer service and identifies areas for improvement.

[0651] Step 8:

[0652] The server receives feedback sent from the terminal and stores it in a database. It then updates the training program based on the collected feedback. For example, if feedback indicates that "the questions about Python were too difficult," "adjusting the difficulty of questions" will be added to the next training program. The updated training program is saved as instructional items for improving staff skills. The input data is the collected feedback, and the output data is the updated training program.

[0653] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0654] This invention relates to a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. This system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs. Furthermore, by incorporating an emotion engine, it has the function of recognizing and analyzing the emotions of interviewers and candidates in real time. Specific embodiments for carrying out this invention are described below.

[0655] 1. System Overview

[0656] This system is equipped with programs to perform the following functions:

[0657] Method for entering candidate information

[0658] Means for analyzing the information of entered candidates

[0659] A means of generating customized interview questions based on analysis results.

[0660] A method for generating follow-up emails after an interview.

[0661] A means of collecting feedback from interviewers and updating the training program based on that feedback.

[0662] A means of combining an emotion engine that recognizes the emotions of interviewers and candidates.

[0663] A means of analyzing emotional data and providing real-time feedback to interviewers based on that data.

[0664] A means of customizing training programs based on emotional data

[0665] 2. Data Entry and Analysis

[0666] Users (interviewers or HR personnel) upload candidates' resumes and work histories to the system. File uploads are done via a web interface or a dedicated desktop application. The terminal temporarily stores these files in local storage and then transfers them to the server.

[0667] The server analyzes the received files using natural language processing techniques. Specifically, it uses OCR technology to extract text and identify important skill sets and career paths. The analyzed data is stored in a database.

[0668] 3. Generating Question and Follow-up Emails

[0669] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. This allows for the automatic creation of questions tailored to the candidate's specific skill set and career path. The generated list of questions is then provided to the interviewer via their terminal.

[0670] After the interview, the server generates a follow-up email, which is also sent to the interviewer via the terminal for confirmation. The follow-up email includes a thank-you message and notification of the next steps.

[0671] 4. Feedback and Training Program

[0672] After the interview is complete, the user (interviewer) enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations.

[0673] The server analyzes the interviewer's performance based on the collected feedback and updates the training program for the next interview. This training program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[0674] 5. Integration of the Emotional Engine

[0675] The server uses an emotion engine that recognizes emotions from the voice and facial expressions of both the interviewer and the candidate during the interview. The emotion engine collects emotional data from voice tone, changes in facial expression, word choice, etc., and analyzes it in real time.

[0676] This emotional data is fed back to the interviewer in real time via the device, supporting smoother communication during the interview. Furthermore, the emotional data collected after the interview is used to customize the interviewer's training program.

[0677] Specific example

[0678] For example, if a candidate's resume is uploaded and that candidate has "Python development experience," "data analysis skills," and "project management experience," the server will automatically generate interview questions like the following.

[0679] "What was the most challenging aspect of your project using Python?"

[0680] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0681] "What lessons have you learned from your project management experience?"

[0682] During the interview, the server uses an emotion engine to analyze the emotions of both the interviewer and the candidate in real time, sending feedback to the terminal such as, "The candidate is nervous about this question." This allows the interviewer to adjust how they ask questions.

[0683] The follow-up email sent after the interview will contain content similar to the following.

[0684] "Thank you for participating in the interview. We will contact you regarding the next steps shortly."

[0685] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[0686] The following describes the processing flow.

[0687] Step 1:

[0688] Users upload candidates' resumes and work histories to the system. Users can drag and drop files or select them from a file selection dialog using a web interface or a dedicated desktop application.

[0689] Step 2:

[0690] The device temporarily saves the uploaded file to local storage and then transfers the file to the server. During this process, an upload progress bar is displayed to inform the user of the progress.

[0691] Step 3:

[0692] The server receives the file sent from the terminal and stores it in the initial database. Next, it starts the process of converting the file to a text format according to its format.

[0693] Step 4:

[0694] The server extracts the text content from the resume using OCR (Optical Character Recognition) technology and text extraction libraries tailored to the file format.

[0695] Step 5:

[0696] The server uses natural language processing (NLP) technology to analyze and extract important information such as skill sets, years of experience, educational background, and work history from the extracted text. The analysis results are stored in a database.

[0697] Step 6:

[0698] The terminal visualizes the analyzed data and displays a list of skill keywords and the candidate's career path on the screen. This allows interviewers to see which skills and experiences they should focus on evaluating.

[0699] Step 7:

[0700] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. The questions are generated to correspond to the candidate's specific skill set and career path.

[0701] Step 8:

[0702] The terminal displays a generated list of questions to the interviewer. The interviewer can select appropriate questions from the displayed list and edit them as needed.

[0703] Step 9:

[0704] The server activates an emotion engine to recognize emotions from the voices and facial expressions of both the interviewer and the candidate during the interview. The emotion engine collects emotional data from voice tone, changes in facial expression, word choice, etc., and analyzes it in real time.

[0705] Step 10:

[0706] The device provides real-time feedback to the interviewer based on emotional data obtained from the emotion engine. This feedback includes specific advice such as, "The candidate is nervous," or "The interviewer should help the candidate relax more."

[0707] Step 11:

[0708] The user (interviewer) conducts interviews with candidates using questions provided by the system. Data and observations obtained during the interview process are then entered into the system.

[0709] Step 12:

[0710] The server automatically generates follow-up emails to send after the interview using an AI model. These follow-up emails include expressions of gratitude and notifications for the next steps.

[0711] Step 13:

[0712] The device displays the contents of the follow-up email to the interviewer, informing them that they can review and revise it.

[0713] Step 14:

[0714] After the interview, the user (interviewer) uses a feedback form to evaluate the quality of the interview and the candidate's responses. The evaluation results are entered into the system.

[0715] Step 15:

[0716] The server receives feedback data and analyzes the interviewer's performance. Based on this analysis, it updates the training program for the next interview.

[0717] Step 16:

[0718] The server analyzes the emotional data collected by the emotion engine and customizes the interviewer's training program based on that analysis. This allows interviewers to receive specific advice on how to improve their emotional recognition and communication skills.

[0719] Step 17:

[0720] The device displays updated training programs and feedback results to the interviewer, providing information to help them prepare for the next interview.

[0721] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[0722] (Example 2)

[0723] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0724] Traditional recruitment processes lacked the ability to generate appropriate questions and provide real-time feedback to facilitate smooth communication between interviewers and applicants. Furthermore, sending follow-up emails after interviews and updating interviewer training programs were often done manually, which was inefficient. Additionally, the lack of automated recognition and feedback on emotions during interviews made it difficult to improve the quality of communication.

[0725] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0726] In this invention, the server includes means for inputting candidate information, means for analyzing the input candidate information and identifying the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, means for generating follow-up emails to be sent after the interview, means for collecting interviewer feedback and updating the interviewer's training program, means for combining an emotion engine that recognizes the emotions of the interviewer and candidate during the interview, means for analyzing the emotion data and providing real-time feedback to the interviewer based on it, and means for customizing the training program based on the emotion data. This makes it possible to improve the quality of communication between interviewers and applicants and to increase the efficiency and effectiveness of the entire recruitment process.

[0727] "Means of inputting candidate information" refers to the interface or device used by interviewers or HR personnel to input information such as candidates' resumes and work histories into a system.

[0728] "Means of analyzing input candidate information to identify candidates' skill sets and career paths" refers to the technologies and processes used by a system to analyze information provided by candidates using natural language processing techniques and extract important skills and work experience.

[0729] "Means for generating customized interview questions based on analyzed information" refers to technologies and processes that use a generation AI model to automatically create individually tailored interview questions based on the analysis results.

[0730] "Methods for generating follow-up emails to send after an interview" refers to the technology or process for automatically generating and sending emails containing expressions of gratitude and notifications regarding the next steps after an interview has concluded.

[0731] "Means of collecting interviewer feedback and updating interviewer training programs" refers to the techniques and processes for collecting feedback information from interviewers after interviews and using that information to continuously update training programs to improve interviewers' skills and knowledge.

[0732] "Means of combining an emotion engine to recognize the emotions of the interviewer and candidate during the interview" refers to a function that implements emotion recognition technology used to analyze emotions from voice, facial expressions, etc., during the interview.

[0733] "Means of analyzing emotional data and providing real-time feedback to interviewers based on that data" refers to technologies and processes that analyze collected emotional data, notify the interviewer of the analysis results in real time, and support the interviewer's response.

[0734] "Methods for customizing training programs based on emotional data" refers to techniques and processes for analyzing emotional data collected during interviews and using the results to customize and improve the interviewer's training program.

[0735] This invention is a system designed to improve communication between interviewers and applicants in a company's recruitment process, while simultaneously increasing the offer acceptance rate. This system functions by combining multiple software and hardware elements. Specific embodiments for carrying out this invention are described below.

[0736] 1. Data entry and temporary storage

[0737] Users upload candidate resumes and CVs to the system using a web interface or a dedicated desktop application. This involves selecting files and clicking an upload button. The terminal temporarily saves the uploaded files to local storage and then transfers them to the server.

[0738] The hardware used includes personal computers, tablets, and smartphones used by the user. The software includes web browsers and dedicated desktop applications.

[0739] 2. Transferring and analyzing data to the server

[0740] The device transfers saved files to the server using a secure communication protocol (e.g., HTTPS). The server extracts text data from the uploaded files using OCR (Optical Character Recognition) technology. This text data is then analyzed using natural language processing (NLP) technology to identify important skill sets and career paths.

[0741] Specific software includes libraries for using OCR technology (e.g., Tesseract) and frameworks using NLP technology (e.g., NLTK, spaCy). The analysis results are stored in a database.

[0742] 3. Generating interview questions and follow-up emails

[0743] The server uses a generative AI model (e.g., OpenAI's GPT, BERT) to generate customized interview questions based on the analysis results. The generated list of questions is stored in a database, and a notification is sent to the terminal. The terminal displays the received notification to the interviewer, who can then view the list of questions via a web interface or application.

[0744] After the interview, the server automatically generates a follow-up email and requests confirmation from the interviewer via the terminal. This email contains an expression of gratitude and information about the next steps.

[0745] For example, if a candidate has "Python development experience," "data analysis skills," and "project management experience," the server will generate interview questions like the following:

[0746] "What was the most challenging aspect of your project using Python?"

[0747] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0748] "What lessons have you learned from your project management experience?"

[0749] 4. Gathering feedback and updating the training program

[0750] After the interview, the user enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and an overall evaluation.

[0751] The server analyzes the collected feedback and stores it in a database as interviewer performance data. It then updates the training program for the next interview. This program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[0752] 5. Integration of the emotion engine and real-time feedback

[0753] The server uses an emotion engine to collect voice and facial expression data from interviewers and candidates in real time during interviews. The emotion engine estimates and analyzes emotional states based on voice tone, changes in facial expressions, and word choice.

[0754] The server sends the analysis results to the terminal in real time and provides feedback to the interviewer. Specific feedback may include comments such as, "The candidate appears nervous about this question."

[0755] Example of a prompt

[0756] You can use the following prompt statements to input data into the generative AI model and obtain the appropriate output.

[0757] "Analyze the next candidate's resume and generate customized interview questions."

[0758] "Please update the training program based on the interview feedback."

[0759] "Analyze the emotional data during the interview and provide real-time feedback."

[0760] In this way, the system can automate a series of processes, aiming to improve the quality of interviews and the offer acceptance rate.

[0761] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0762] Step 1:

[0763] Users upload candidate resumes and CVs using a web interface or a dedicated desktop application. The user selects the file and clicks the upload button. The input is the candidate's resume or CV, and the output is the uploaded file. Specifically, a file selection dialog opens, the user selects the file, and clicks the "Upload" button.

[0764] Step 2:

[0765] The terminal temporarily saves uploaded files to local storage. The input is the uploaded file, and the output is the file saved in local storage. Specifically, the system retrieves basic file information (filename, size, upload time, etc.) and saves the file to local storage.

[0766] Step 3:

[0767] The terminal transfers saved files to the server using a secure communication protocol (e.g., HTTPS). The input is a file stored in local storage, and the output is a file transferred to the server. Specifically, the terminal sends a secure HTTP request to the server and transfers the file data.

[0768] Step 4:

[0769] The server analyzes the received file using OCR technology. The input is the file uploaded to the server, and the output is the analyzed text data. Specifically, the server uses the Tesseract library to extract the text data from the file and saves it to a database.

[0770] Step 5:

[0771] The server analyzes text data extracted using natural language processing technology. The input is text data extracted by OCR, and the output is the identification of candidates' skill sets and career paths. Specifically, the server uses NLTK and spaCy to analyze the text data and identify important skill sets and work history information.

[0772] Step 6:

[0773] The server generates customized interview questions using a generative AI model based on the analysis results. The input is the analyzed candidate information, and the output is the generated list of interview questions. Specifically, the server inputs prompt sentences into the generative AI model (e.g., OpenAI's GPT) and generates the list of questions.

[0774] Step 7:

[0775] The server saves the generated question list to a database and sends a notification to the terminal. The input is the generated interview question list, and the output is the notification message. Specifically, the server saves the question list to the database and sends a notification message to the terminal.

[0776] Step 8:

[0777] The terminal displays notifications received from the server to the interviewer. The input is the notification message from the server, and the output is a display of the question list that the interviewer can view. Specifically, the terminal displays the notification message in the user interface, and clicking it displays the details of the question list.

[0778] Step 9:

[0779] The server automatically generates a follow-up email after the interview is completed. The input is the interview completion information, and the output is the follow-up email. Specifically, the server automatically generates the email content based on a follow-up email template and sends it to the terminal.

[0780] Step 10:

[0781] The terminal displays the follow-up email received from the server to the interviewer and requests their confirmation of its contents. The input is the received follow-up email, and the output is the follow-up email confirmed by the interviewer. Specifically, the terminal displays the contents of the follow-up email, allowing the interviewer to make corrections or approve it.

[0782] Step 11:

[0783] The user inputs interview feedback into the system interface. The input is the interviewer's feedback, and the output is feedback data. Specifically, the user enters the feedback into the input field and presses the submit button.

[0784] Step 12:

[0785] The server analyzes the collected feedback and updates the training program. The input is the collected feedback data, and the output is the updated training program. Specifically, the server analyzes the feedback data, generates the training content, and saves it to the database.

[0786] Step 13:

[0787] The server uses an emotion engine to collect and analyze emotional data during the interview. Inputs are voice and facial expression data, and output is the analyzed emotional data. Specifically, the server uses emotion recognition technology to analyze the emotional data in real time and provides it to the interviewer as feedback.

[0788] Step 14:

[0789] The server provides real-time feedback to the interviewer based on emotional data. The input is analyzed emotional data, and the output is a feedback message. Specifically, the server analyzes the emotional data and sends it to the interviewer's terminal so that they can receive visualized feedback.

[0790] Step 15:

[0791] The server customizes the training program based on emotional data. The input is emotional data, and the output is the customized training program. Specifically, the server analyzes the emotional data, optimizes the training content for the interviewer, and stores it in a database.

[0792] (Application Example 2)

[0793] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0794] This invention relates to a system for improving efficiency and accuracy in conventional recruitment processes and the work performance of factory robots. Conventional recruitment processes often rely heavily on the subjective opinions and experience of interviewers, resulting in lower offer acceptance rates. Similarly, in factory robot operations, delays in operator feedback can reduce efficiency and increase the likelihood of errors. This invention aims to solve these problems and improve the quality of interviews and factory work.

[0795] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0796] In this invention, the server includes means for analyzing candidate information and identifying the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, and means equipped with an emotion engine for collecting data in real time and providing feedback to the operator. This enables the provision of objective and customized questions to the interviewer, and improves work efficiency and accuracy by providing real-time feedback based on robotic and human emotion data.

[0797] A "candidate" is an individual who possesses the necessary information and skills to apply for a specific position.

[0798] "Means for analyzing information" refers to a device or program that has the function of identifying specific attributes or patterns based on input data and generating the necessary output based on those attributes or patterns.

[0799] A "skill set" refers to the abilities and experience required to perform a specific job or role.

[0800] A "career path" is a diagram or outline that shows an individual's work history and professional experience, illustrating the future progression of their career.

[0801] "Means for generating customized interview questions" refers to a device or program that automatically creates questions optimized for individual candidates based on analysis results.

[0802] A "follow-up email" is an email sent to a candidate after an interview, containing a thank you and information about the next steps.

[0803] An "interviewer" is an individual responsible for conducting interviews with candidates and evaluating their performance.

[0804] A "training program" is a curriculum of education and training designed to improve the skills and performance of interviewers.

[0805] "Means of collecting data in real time" refers to devices or programs that have the function of immediately acquiring and analyzing data generated on-site.

[0806] An "emotion engine" is a device or program that recognizes and analyzes emotions based on a person's facial expressions, tone of voice, and choice of words.

[0807] An "operator" is an individual responsible for operating machinery or equipment.

[0808] "Means of providing feedback" refers to a device or program that notifies operators or users in real time of suggestions for improvement or precautions based on analysis results.

[0809] 1. System Configuration

[0810] This invention provides a system for improving the work performance of factory robots. The system includes the following components:

[0811] 1. Means of entering candidate information

[0812] 2. A means of analyzing the entered candidate information to identify the candidate's skill set and career path.

[0813] 3. Means for generating customized interview questions based on analyzed information

[0814] 4. A method for generating follow-up emails to send after the interview has concluded.

[0815] 5. Means of collecting interviewer feedback and updating interviewer training programs.

[0816] 6. A system equipped with an emotion engine to collect data in real time and provide feedback to operators.

[0817] 7. A means of analyzing robot and human emotional data and providing real-time feedback to the operator based on the results.

[0818] 2. Program Processing

[0819] The program for realizing this invention performs the following processes:

[0820] Hardware and software usage

[0821] Camera: A video capture device is used to monitor the operation of factory robots. The camera captures the robot's movements as video in real time.

[0822] Emotion Recognition Model: Use an emotion recognition model trained with TensorFlow. Specifically, use the model named emotion_model.h5.

[0823] Face Recognition: Faces are detected using OpenCV's Haar Cascade Classifier. This allows for the analysis of the emotions of the operator and other participants during robot operation.

[0824] Natural Language Processing: Use the TextBlob library to analyze and process text data generated based on the analysis results.

[0825] Sending emails: Use the smtplib library to send notification emails to operators.

[0826] Specific example

[0827] As a concrete example, if a factory robot detects abnormal behavior during operation, the system will operate as follows:

[0828] First, cameras monitor the robot's movements within the factory and acquire video data in real time. The system detects the operator's facial expressions using a Haar cascade classifier and analyzes their emotions using an emotion recognition model (TensorFlow model).

[0829] For example, if an operator expresses emotions such as "surprise" or "anger," the system analyzes that emotion data and determines that a specific anomaly has occurred. In this case, a feedback message like the following is generated:

[0830] "An abnormality has been detected in the robot's operation. Please be careful."

[0831] This feedback message is displayed on the operator's screen in real time and is also notified via email. The smtplib library is used for sending emails.

[0832] Example of a prompt

[0833] Here are some specific examples of prompt statements to input into a generative AI model:

[0834] "Please prepare a detailed report on any issues related to the robot's operation. The report should include the type of anomaly detected, the countermeasures taken, and measures to prevent recurrence."

[0835] These components and processing procedures make it possible to improve the operational performance of factory robots and provide an efficient and safe working environment.

[0836] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0837] Step 1:

[0838] Capture images with a camera

[0839] The terminal acquires real-time video footage of robots and operators operating within the factory from cameras. The input is video data from the cameras, which is processed as a video stream. The output is raw video data.

[0840] Step 2:

[0841] Face detection

[0842] The server detects faces from the acquired video data using OpenCV's Haar cascade classifier. The input is the video data acquired in step 1, which is processed to extract specific areas (faces). The output is face coordinate (x, y, w, h) data.

[0843] Step 3:

[0844] Facial cropping and pre-processing

[0845] The device extracts the face portion from the video data based on the detected face coordinate data and converts it into a format that can be input into the emotion recognition model. The input consists of face coordinate data and raw video data, which are used to extract the face image, resize it, and convert it to grayscale. The output is a pre-processed face image.

[0846] Step 4:

[0847] Recognition of emotions

[0848] The server inputs pre-processed facial images into a TensorFlow emotion recognition model to analyze emotions. The input is pre-processed facial images, and the model predicts emotions. The output is emotion prediction data (labels such as anger, surprise, sadness, etc.).

[0849] Step 5:

[0850] Detection of abnormal operation

[0851] The server analyzes the output data of the emotion recognition model and recognizes abnormal emotions (e.g., anger) as abnormal behavior. The input is emotion prediction data, which is analyzed based on rules for determining whether an abnormality exists. The output is a flag indicating whether an abnormal behavior occurred.

[0852] Step 6:

[0853] Generating feedback messages

[0854] If an abnormal operation is detected, the server uses a generative AI model to generate a feedback message for the operator. The input is an abnormal operation flag and a prompt statement (e.g., "Please prepare a detailed report on the robot's operation issue"), and the AI ​​model generates an appropriate feedback message. The output is the feedback message text.

[0855] Step 7:

[0856] Real-time display of feedback

[0857] The terminal displays the generated feedback message on the operator's display in real time. The input is the feedback message text, and its content is transmitted to the operator via the display device. The output is visual feedback information.

[0858] Step 8:

[0859] Sending notification emails

[0860] The server generates a notification email containing a feedback message and sends it to the operator's email address. The input is the feedback message text and the operator's email address, and the email is sent using the smtplib library. The output is the sent notification email.

[0861] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0862] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0863] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0864] [Third Embodiment]

[0865] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0866] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0867] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0868] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0869] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0870] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0871] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0872] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0873] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0874] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0875] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0876] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0877] This invention relates to a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. This system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs. Specific embodiments for carrying out this invention are described below.

[0878] 1. System Overview

[0879] This system is equipped with programs to perform the following functions:

[0880] Method for entering candidate information

[0881] Means for analyzing the information of entered candidates

[0882] A means of generating customized interview questions based on analysis results.

[0883] A method for generating follow-up emails after an interview.

[0884] A means of collecting feedback from interviewers and updating the training program based on that feedback.

[0885] 2. Data Entry and Analysis

[0886] Users (interviewers or HR personnel) upload candidates' resumes and work histories to the system. File uploads are done via a web interface or a dedicated desktop application. The terminal temporarily stores these files in local storage and then transfers them to the server.

[0887] The server analyzes the received files using natural language processing techniques. Specifically, it uses OCR technology to extract text and identify important skill sets and career paths. The analyzed data is stored in a database.

[0888] 3. Generating Question and Follow-up Emails

[0889] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. This allows for the automatic creation of questions tailored to the candidate's specific skill set and career path. The generated list of questions is then provided to the interviewer via their terminal.

[0890] After the interview, the server generates a follow-up email, which is also sent to the interviewer via the terminal for confirmation. The follow-up email includes a thank-you message and notification of the next steps.

[0891] 4. Feedback and Training Program

[0892] After the interview is complete, the user (interviewer) enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations.

[0893] The server analyzes the interviewer's performance based on the collected feedback and updates the training program for the next interview. This training program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[0894] Specific example

[0895] For example, if a candidate's resume is uploaded and that candidate has "Python development experience," "data analysis skills," and "project management experience," the server will automatically generate interview questions like the following.

[0896] "What was the most challenging aspect of your project using Python?"

[0897] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[0898] "What lessons have you learned from your project management experience?"

[0899] Furthermore, the follow-up email sent after the interview will be generated with content similar to the following.

[0900] "Thank you for participating in the interview. We will contact you regarding the next steps shortly."

[0901] In this way, the system can improve the quality of interviews and increase the offer acceptance rate.

[0902] The following describes the processing flow.

[0903] Step 1:

[0904] Users can upload candidates' resumes and work histories by dragging and dropping them into the system's upload form, or by selecting them from a file selection dialog.

[0905] Step 2:

[0906] The device temporarily saves the uploaded file to local storage and displays a progress bar to the user that says, "Sending data to the server."

[0907] Step 3:

[0908] The server receives the file sent from the terminal and stores it in the initial database. After that, it starts processing to convert the file contents into text format.

[0909] Step 4:

[0910] The server uses OCR (Optical Character Recognition) technology and libraries that extract text according to the file format to obtain the text content of the resume.

[0911] Step 5:

[0912] The server uses a natural language processing (NLP) model to analyze and extract important information such as skill sets, years of experience, educational background, and work history from the extracted text. The analysis results are stored in a database.

[0913] Step 6:

[0914] The terminal visualizes the analyzed data and displays a list of skill keywords and the candidate's career path on the screen. This allows interviewers to see which skills and experiences they should focus on evaluating.

[0915] Step 7:

[0916] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. The questions are generated to correspond to the candidate's specific skill set and career path.

[0917] Step 8:

[0918] The terminal displays a generated list of questions to the interviewer. The interviewer can select appropriate questions from the displayed list and edit them as needed.

[0919] Step 9:

[0920] The user conducts interviews with candidates using questions provided by the system. Data and observations obtained during the interview process are then entered into the system.

[0921] Step 10:

[0922] The server automatically generates follow-up emails to send after the interview using an AI model. These follow-up emails include expressions of gratitude and notifications for the next steps.

[0923] Step 11:

[0924] The device displays the contents of the follow-up email to the interviewer, informing them that they can review and revise it.

[0925] Step 12:

[0926] After the interview, the user (interviewer) uses a feedback form to evaluate the quality of the interview and the candidate's responses. The evaluation results are entered into the system.

[0927] Step 13:

[0928] The server receives feedback data and analyzes the interviewer's performance. Based on this analysis, it updates the training program for the next interview.

[0929] Step 14:

[0930] The device displays updated training programs and feedback results to the interviewer, providing information to help them prepare for the next interview.

[0931] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[0932] (Example 1)

[0933] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0934] In a company's recruitment process, it is crucial to streamline communication between interviewers and applicants and increase the offer acceptance rate. However, traditional manual interview preparation and follow-up email creation are time-consuming and labor-intensive, and suffer from inconsistencies in consistency and quality. Furthermore, insufficient interviewer training can lead to low interview quality. A system is needed to address these challenges and realize a more effective and efficient recruitment process.

[0935] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0936] In this invention, the server includes means for the user to input candidate information, means for the terminal to temporarily store the entered candidate information in local storage and transfer it to the server, means for the server to analyze the received candidate information and identify the candidate's skill set and career path, means for generating customized interview questions using a generative AI model based on the analyzed information, means for generating follow-up emails using the generative AI model after the interview is completed, and means for collecting interviewer feedback from the user and for the server to update the training program based on that feedback. This enables the system to streamline interview preparation, automate follow-up emails, and optimize interviewer training.

[0937] A "user" is a person or person who uses the system to input candidate information or interviewer feedback.

[0938] A "terminal" refers to a device such as a computer or tablet used by a user to temporarily store candidate information and transfer it to a server.

[0939] A "server" is the central hub of the entire system, a computer system that performs tasks such as analyzing candidate information, generating customized interview questions, creating follow-up emails, and updating training programs.

[0940] "Candidate information" refers to the content of resumes and work experience summaries obtained during the recruitment process.

[0941] "Local storage" refers to the area on a device where data is temporarily stored.

[0942] "Transfer" refers to the process of sending candidate information from a terminal to a server.

[0943] "Analysis" refers to the process of using natural language processing technology to analyze the received candidate information and identify their skill set and career path.

[0944] A "generative AI model" is an artificial intelligence model used to automatically generate customized interview questions and follow-up emails.

[0945] "Customized interview questions" are specific questions created by a generative AI model based on analyzed candidate information.

[0946] A "follow-up email" is an email sent to a candidate after an interview to express gratitude and provide information about the next steps.

[0947] "Feedback" refers to the opinions and comments that interviewers enter after an interview regarding the candidate's responses, the appropriateness of the questions, and other observations.

[0948] A "training program" is educational content that the server updates based on feedback to improve the interviewer's performance.

[0949] Modes for carrying out the invention

[0950] This invention is a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. The system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs.

[0951] Hardware and software to be used

[0952] hardware

[0953] Server: This is the central hub of the entire system, performing data analysis, generating questions using generative AI models, creating follow-up emails, and updating training programs.

[0954] Device: A device such as a computer or tablet used by the user (interviewer or HR representative).

[0955] software

[0956] Web Interface: Users upload candidate information via a web page using HTML / CSS / JavaScript.

[0957] OCR technology: Use Tesseract OCR to extract text from candidates' resumes and work histories.

[0958] Natural Language Processing (NLP) Libraries: Use Python-based NLP libraries (e.g., spaCy or NLTK) to analyze text and identify skill sets and career paths.

[0959] Generative AI Model: Uses OpenAI's GPT-3 and other technologies to generate customized interview questions and follow-up emails.

[0960] Database: We use MySQL or PostgreSQL to store analysis results and feedback data.

[0961] Specific operation of the system

[0962] 1. Data entry

[0963] Users upload resumes and work histories of interview candidates to the system. Uploads can be made via a web interface or a dedicated desktop application. For example, a user can select a file from their PC and press the upload button, and the file will be sent to the server.

[0964] 2. Data Analysis

[0965] The terminal temporarily saves uploaded files to local storage and then transfers them to the server. The server uses Tesseract OCR to extract text from the files and analyzes the text using Python's NLP library to identify important skill sets and career paths. The analyzed data is stored in a database and used for subsequent processing.

[0966] 3. Generating interview questions

[0967] Based on the analysis results, the server uses a generation AI model to generate customized interview questions. As a specific example, enter the following prompt text:

[0968] Skills extracted from the candidate's resume:

[0969] Python development experience

[0970] Data analysis skills

[0971] Project management experience

[0972] Based on the above skill set, generate questions that interviewers should ask candidates.

[0973] Based on this prompt, the generative AI model generates the following question.

[0974] 1. "What was the most challenging aspect of your project using Python?"

[0975] 2. "Please tell us about the specific tools and techniques you used when performing data analysis."

[0976] 3. "What lessons have you learned from your project management experience?"

[0977] 4. Conducting interviews and generating follow-up emails

[0978] The user conducts interviews with candidates using a list of questions provided through their device. After the interview, the server generates a follow-up email using a generative AI model. The follow-up email includes an expression of gratitude and a notification about the next steps. For example, it might contain the following:

[0979] Subject: Thank you for participating in the interview.

[0980] Main text:

[0981] [Candidate Name],

[0982] Thank you very much for participating in our interview. We will contact you regarding the next steps shortly. If you have any questions, please do not hesitate to contact us.

[0983] Thank you for your cooperation.

[0984] [Company Name] Recruitment Team

[0985] 5. Gathering feedback and updating the training program

[0986] After an interview, the user (interviewer) enters feedback through the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations. The server analyzes the collected feedback and evaluates the interviewer's performance. A training program for the next interview is automatically updated.

[0987] In this way, this system can improve the quality of interviews and increase the offer acceptance rate.

[0988] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0989] Step 1: User uploads candidate information.

[0990] Users upload resumes and work histories of interview candidates to the system. Input consists of candidate document files (PDF or Word files), primarily using a web interface or a dedicated desktop application. Specifically, users click a file selection button, select a file from their PC, and click the upload button. This action sends the file to the server.

[0991] Step 2: Saving files locally on the terminal and transferring them to the server.

[0992] The terminal temporarily saves the user-uploaded file to local storage. The file is then transferred to the specified server endpoint using an HTTP POST request. The input is the file uploaded by the user, and the output is the file transferred to the server.

[0993] Step 3: Server analyzes candidate information

[0994] The server analyzes the received file. First, it extracts text from the file using OCR technology (e.g., Tesseract OCR). This text extraction result is then analyzed using a Python NLP library (e.g., spaCy) to identify important skill sets and career paths. The input is the file transferred to the server, and the output is the analyzed information on the candidate's skill sets and career paths. Specifically, the server performs OCR processing and converts the resulting text data into structured data using natural language processing techniques.

[0995] Step 4: Server generates interview questions

[0996] The server generates customized interview questions using a generative AI model (e.g., OpenAI's GPT-3) based on the analysis results. The prompt text is input to the generative AI model, and the generated question list is obtained. The input is the analyzed candidate information and prompt text, and the output is a list of generated interview questions. The following prompt text is used as a concrete example.

[0997] Skills extracted from the candidate's resume:

[0998] Python development experience

[0999] Data analysis skills

[1000] Project management experience

[1001] Based on the above skill set, generate questions that interviewers should ask candidates.

[1002] The generative AI model generates a list of questions based on this.

[1003] Step 5: Conducting interviews with users

[1004] The user conducts interviews with candidates using a list of interview questions provided via the terminal. In this step, the user conducts actual interviews based on the generated question list and evaluates the candidates' responses. The input is the generated interview question list, and the output is the interview results.

[1005] Step 6: Server generates follow-up email

[1006] After the interview ends, the server uses a generative AI model to generate a follow-up email. The follow-up email includes an expression of gratitude and notification of the next steps. The input is the interview completion information, and the output is the generated follow-up email. As a specific example, the following follow-up email is generated:

[1007] Subject: Thank you for participating in the interview.

[1008] Main text:

[1009] [Candidate Name],

[1010] Thank you very much for participating in our interview. We will contact you regarding the next steps shortly. If you have any questions, please do not hesitate to contact us.

[1011] Thank you for your cooperation.

[1012] [Company Name] Recruitment Team

[1013] Step 7: User enters interview feedback.

[1014] After the interview, the user enters feedback through the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations. The input is interview feedback information, and the output is feedback data stored on the server.

[1015] Step 8: Server updates the training program

[1016] The server analyzes the collected feedback, evaluates the interviewer's performance, and updates the training program for the next interview. The input is feedback data, and the output is the updated training program. The server integrates this training program with other feedback data and interview data to provide optimal improvement suggestions.

[1017] (Application Example 1)

[1018] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1019] The goal is to improve the quality of customer service in physical stores and increase customer satisfaction. In particular, there is a need for a system that efficiently generates customized customer service questions based on customer interests and purchase history, automatically generates follow-up emails after customer service, and collects staff feedback and updates training programs.

[1020] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1021] In this invention, the server includes means for inputting customer information, means for analyzing the information to identify customer interests and purchase history, means for generating customized customer service questions, means for generating follow-up emails after the customer service is completed, and means for collecting staff feedback and updating training programs. This makes it possible to provide high-quality customer service tailored to the individual needs of customers and improve customer satisfaction.

[1022] A "customer" is a person who visits a commercial facility or service provider, and is the provider of information that identifies their interests and purchasing history.

[1023] "Information analysis" refers to the act of processing input data and extracting meaning and patterns from it, and in this invention, it is used to identify customer interests and purchase history.

[1024] "Customized customer service questions" are questions that are dynamically generated based on each customer's interests and purchase history, and are designed to provide individualized service.

[1025] A "follow-up email" is an email sent to a customer after the service has concluded, containing a message of gratitude and notification of the next steps.

[1026] "Staff" refers to employees who are engaged in providing services to customers in a physical store, and are the subjects of feedback and training programs.

[1027] "Feedback" refers to the opinions and evaluations that staff members have regarding customer service, and the training program is updated based on this feedback.

[1028] A "training program" refers to educational content aimed at improving staff customer service skills, and it is optimized based on feedback.

[1029] "Natural language processing technology" refers to a set of technologies used in the fields of computer science and artificial intelligence to understand and generate natural language, and is used in information analysis.

[1030] "Uploading data via the internet" refers to the act of transmitting customer information or other data to a system using an online network.

[1031] Modes for carrying out the invention

[1032] 1. System Program

[1033] Explain the program's processing in natural language.

[1034] This system is designed to provide customized customer service in physical stores. The specific processing flow is described below.

[1035] Customer information input and analysis

[1036] Users (staff) input basic customer information and purchase history into the system using devices such as smartphones and smart glasses. The devices transmit this information to the server via the internet. The server analyzes the received data using natural language processing techniques to identify customer interests and purchase history. This analysis uses natural language processing libraries such as Spacy and generative AI models such as GPT-3.

[1037] Generating customized customer service questions

[1038] The server generates prompts based on the analysis results, which use a generative AI model to generate customized customer service questions. The generated questions are then provided to the staff via the terminal. For example, for a customer with "Python development experience," "data analysis skills," and "project management experience," the following questions might be generated:

[1039] "What was the most challenging aspect of your project using Python?"

[1040] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1041] "What lessons have you learned from your project management experience?"

[1042] Generating follow-up emails

[1043] After the customer service session ends, the server automatically generates a follow-up email using a generation AI model. This email includes a message of gratitude and encourages the customer to visit again. The generated email is reviewed by staff via their terminals and then sent to the customer.

[1044] Example: "Thank you for visiting our store today. We look forward to your next visit."

[1045] Feedback collection and training program updates

[1046] Users (staff) input feedback about customer service using a terminal. This feedback includes customer reactions and the appropriateness of questions asked. The server stores this feedback information in a database and uses it to update staff training programs. This training program is optimized based on the collected feedback and reflected in future customer service interactions.

[1047] Specific example

[1048] Let's say a customer comes into the store and has "Python development experience," "data analysis skills," and "project management experience." The user (staff) enters the customer's basic information on their smartphone. The server analyzes this information and uses a generative AI model to generate customized questions like the following.

[1049] "What was the most challenging aspect of your project using Python?"

[1050] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1051] "What lessons have you learned from your project management experience?"

[1052] After serving a customer, the server automatically generates a follow-up email using a generative AI model, such as the following:

[1053] "Thank you for visiting our store today. We look forward to your next visit."

[1054] Example of a prompt

[1055] Prompt: "Generate customer service questions for a person with experience in Python development, data analysis, and project management."

[1056] In this way, this system can improve the quality of customer service in physical stores and increase customer satisfaction.

[1057] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1058] Step 1:

[1059] Users input basic customer information and purchase history using devices such as smartphones or smart glasses. Specifically, they enter data such as the customer's name, age, and past purchases into input fields. This data is transmitted from the device to the server via the internet. The input data consists of the customer's basic information and purchase history.

[1060] Step 2:

[1061] The server receives basic customer information and purchase history transmitted from the terminal. Next, natural language processing techniques (such as Spacy) are used to analyze this data and identify customer interests and purchasing patterns. As a result of the data analysis, the customer's areas of interest and purchasing tendencies are identified. The analyzed information includes the customer's interests and purchase history.

[1062] Step 3:

[1063] The server generates prompts based on the analysis results. For example, if "Python development experience," "data analysis skills," and "project management experience" are identified, the prompt might be "Generate customer service questions for a person with experience in Python development, data analysis, and project management." The generated prompts serve as input data for generating customized customer service questions.

[1064] Step 4:

[1065] The server uses a generative AI model (such as GPT-3) to take prompt text as input and generate customized customer service questions. Specific questions include, "What was the most challenging aspect of your project using Python?" The generated list of questions is sent to the terminal as customized customer service questions and provided to the staff. The prompt text is used as input data, and the generated customer service questions are used as output data.

[1066] Step 5:

[1067] The user (staff) interacts with the customer based on customized questions displayed on the terminal. Information obtained during the interaction can also be added to the input fields as needed. The entered data constitutes additional customer information.

[1068] Step 6:

[1069] After the customer service is complete, the server automatically generates a follow-up email using a generation AI model. For example, a follow-up email with content such as, "Thank you for visiting our store today. We look forward to your next visit," is generated. The generated follow-up email is sent to the customer's terminal as a follow-up message. The customer service results are used as input data, and the generated follow-up email is used as output data.

[1070] Step 7:

[1071] Users (staff) input feedback on customer service using a terminal. This feedback includes customer reactions and the appropriateness of questions asked. The input feedback data is sent to a server via the internet. The feedback serves as an evaluation of the staff's customer service and identifies areas for improvement.

[1072] Step 8:

[1073] The server receives feedback sent from the terminal and stores it in a database. It then updates the training program based on the collected feedback. For example, if feedback indicates that "the questions about Python were too difficult," "adjusting the difficulty of questions" will be added to the next training program. The updated training program is saved as instructional items for improving staff skills. The input data is the collected feedback, and the output data is the updated training program.

[1074] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1075] This invention relates to a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. This system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs. Furthermore, by incorporating an emotion engine, it has the function of recognizing and analyzing the emotions of interviewers and candidates in real time. Specific embodiments for carrying out this invention are described below.

[1076] 1. System Overview

[1077] This system is equipped with programs to perform the following functions:

[1078] Method for entering candidate information

[1079] Means for analyzing the information of entered candidates

[1080] A means of generating customized interview questions based on analysis results.

[1081] A method for generating follow-up emails after an interview.

[1082] A means of collecting feedback from interviewers and updating the training program based on that feedback.

[1083] A means of combining an emotion engine that recognizes the emotions of interviewers and candidates.

[1084] A means of analyzing emotional data and providing real-time feedback to interviewers based on that data.

[1085] A means of customizing training programs based on emotional data

[1086] 2. Data Entry and Analysis

[1087] Users (interviewers or HR personnel) upload candidates' resumes and work histories to the system. File uploads are done via a web interface or a dedicated desktop application. The terminal temporarily stores these files in local storage and then transfers them to the server.

[1088] The server analyzes the received files using natural language processing techniques. Specifically, it uses OCR technology to extract text and identify important skill sets and career paths. The analyzed data is stored in a database.

[1089] 3. Generating Question and Follow-up Emails

[1090] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. This allows for the automatic creation of questions tailored to the candidate's specific skill set and career path. The generated list of questions is then provided to the interviewer via their terminal.

[1091] After the interview, the server generates a follow-up email, which is also sent to the interviewer via the terminal for confirmation. The follow-up email includes a thank-you message and notification of the next steps.

[1092] 4. Feedback and Training Program

[1093] After the interview is complete, the user (interviewer) enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations.

[1094] The server analyzes the interviewer's performance based on the collected feedback and updates the training program for the next interview. This training program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[1095] 5. Integration of the Emotional Engine

[1096] The server uses an emotion engine that recognizes emotions from the voice and facial expressions of both the interviewer and the candidate during the interview. The emotion engine collects emotional data from voice tone, changes in facial expression, word choice, etc., and analyzes it in real time.

[1097] This emotional data is fed back to the interviewer in real time via the device, supporting smoother communication during the interview. Furthermore, the emotional data collected after the interview is used to customize the interviewer's training program.

[1098] Specific example

[1099] For example, if a candidate's resume is uploaded and that candidate has "Python development experience," "data analysis skills," and "project management experience," the server will automatically generate interview questions like the following.

[1100] "What was the most challenging aspect of your project using Python?"

[1101] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1102] "What lessons have you learned from your project management experience?"

[1103] During the interview, the server uses an emotion engine to analyze the emotions of both the interviewer and the candidate in real time, sending feedback to the terminal such as, "The candidate is nervous about this question." This allows the interviewer to adjust how they ask questions.

[1104] The follow-up email sent after the interview will contain content similar to the following.

[1105] "Thank you for participating in the interview. We will contact you regarding the next steps shortly."

[1106] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[1107] The following describes the processing flow.

[1108] Step 1:

[1109] Users upload candidates' resumes and work histories to the system. Users can drag and drop files or select them from a file selection dialog using a web interface or a dedicated desktop application.

[1110] Step 2:

[1111] The device temporarily saves the uploaded file to local storage and then transfers the file to the server. During this process, an upload progress bar is displayed to inform the user of the progress.

[1112] Step 3:

[1113] The server receives the file sent from the terminal and stores it in the initial database. Next, it starts the process of converting the file to a text format according to its format.

[1114] Step 4:

[1115] The server extracts the text content from the resume using OCR (Optical Character Recognition) technology and text extraction libraries tailored to the file format.

[1116] Step 5:

[1117] The server uses natural language processing (NLP) technology to analyze and extract important information such as skill sets, years of experience, educational background, and work history from the extracted text. The analysis results are stored in a database.

[1118] Step 6:

[1119] The terminal visualizes the analyzed data and displays a list of skill keywords and the candidate's career path on the screen. This allows interviewers to see which skills and experiences they should focus on evaluating.

[1120] Step 7:

[1121] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. The questions are generated to correspond to the candidate's specific skill set and career path.

[1122] Step 8:

[1123] The terminal displays a generated list of questions to the interviewer. The interviewer can select appropriate questions from the displayed list and edit them as needed.

[1124] Step 9:

[1125] The server activates an emotion engine to recognize emotions from the voices and facial expressions of both the interviewer and the candidate during the interview. The emotion engine collects emotional data from voice tone, changes in facial expression, word choice, etc., and analyzes it in real time.

[1126] Step 10:

[1127] The device provides real-time feedback to the interviewer based on emotional data obtained from the emotion engine. This feedback includes specific advice such as, "The candidate is nervous," or "The interviewer should help the candidate relax more."

[1128] Step 11:

[1129] The user (interviewer) conducts interviews with candidates using questions provided by the system. Data and observations obtained during the interview process are then entered into the system.

[1130] Step 12:

[1131] The server automatically generates follow-up emails to send after the interview using an AI model. These follow-up emails include expressions of gratitude and notifications for the next steps.

[1132] Step 13:

[1133] The device displays the contents of the follow-up email to the interviewer, informing them that they can review and revise it.

[1134] Step 14:

[1135] After the interview, the user (interviewer) uses a feedback form to evaluate the quality of the interview and the candidate's responses. The evaluation results are entered into the system.

[1136] Step 15:

[1137] The server receives feedback data and analyzes the interviewer's performance. Based on this analysis, it updates the training program for the next interview.

[1138] Step 16:

[1139] The server analyzes the emotional data collected by the emotion engine and customizes the interviewer's training program based on that analysis. This allows interviewers to receive specific advice on how to improve their emotional recognition and communication skills.

[1140] Step 17:

[1141] The device displays updated training programs and feedback results to the interviewer, providing information to help them prepare for the next interview.

[1142] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[1143] (Example 2)

[1144] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1145] Traditional recruitment processes lacked the ability to generate appropriate questions and provide real-time feedback to facilitate smooth communication between interviewers and applicants. Furthermore, sending follow-up emails after interviews and updating interviewer training programs were often done manually, which was inefficient. Additionally, the lack of automated recognition and feedback on emotions during interviews made it difficult to improve the quality of communication.

[1146] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1147] In this invention, the server includes means for inputting candidate information, means for analyzing the input candidate information and identifying the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, means for generating follow-up emails to be sent after the interview, means for collecting interviewer feedback and updating the interviewer's training program, means for combining an emotion engine that recognizes the emotions of the interviewer and candidate during the interview, means for analyzing the emotion data and providing real-time feedback to the interviewer based on it, and means for customizing the training program based on the emotion data. This makes it possible to improve the quality of communication between interviewers and applicants and to increase the efficiency and effectiveness of the entire recruitment process.

[1148] "Means of inputting candidate information" refers to the interface or device used by interviewers or HR personnel to input information such as candidates' resumes and work histories into a system.

[1149] "Means of analyzing input candidate information to identify candidates' skill sets and career paths" refers to the technologies and processes used by a system to analyze information provided by candidates using natural language processing techniques and extract important skills and work experience.

[1150] "Means for generating customized interview questions based on analyzed information" refers to technologies and processes that use a generation AI model to automatically create individually tailored interview questions based on the analysis results.

[1151] "Methods for generating follow-up emails to send after an interview" refers to the technology or process for automatically generating and sending emails containing expressions of gratitude and notifications regarding the next steps after an interview has concluded.

[1152] "Means of collecting interviewer feedback and updating interviewer training programs" refers to the techniques and processes for collecting feedback information from interviewers after interviews and using that information to continuously update training programs to improve interviewers' skills and knowledge.

[1153] "Means of combining an emotion engine to recognize the emotions of the interviewer and candidate during the interview" refers to a function that implements emotion recognition technology used to analyze emotions from voice, facial expressions, etc., during the interview.

[1154] "Means of analyzing emotional data and providing real-time feedback to interviewers based on that data" refers to technologies and processes that analyze collected emotional data, notify the interviewer of the analysis results in real time, and support the interviewer's response.

[1155] "Methods for customizing training programs based on emotional data" refers to techniques and processes for analyzing emotional data collected during interviews and using the results to customize and improve the interviewer's training program.

[1156] This invention is a system designed to improve communication between interviewers and applicants in a company's recruitment process, while simultaneously increasing the offer acceptance rate. This system functions by combining multiple software and hardware elements. Specific embodiments for carrying out this invention are described below.

[1157] 1. Data entry and temporary storage

[1158] Users upload candidate resumes and CVs to the system using a web interface or a dedicated desktop application. This involves selecting files and clicking an upload button. The terminal temporarily saves the uploaded files to local storage and then transfers them to the server.

[1159] The hardware used includes personal computers, tablets, and smartphones used by the user. The software includes web browsers and dedicated desktop applications.

[1160] 2. Transferring and analyzing data to the server

[1161] The device transfers saved files to the server using a secure communication protocol (e.g., HTTPS). The server extracts text data from the uploaded files using OCR (Optical Character Recognition) technology. This text data is then analyzed using natural language processing (NLP) technology to identify important skill sets and career paths.

[1162] Specific software includes libraries for using OCR technology (e.g., Tesseract) and frameworks using NLP technology (e.g., NLTK, spaCy). The analysis results are stored in a database.

[1163] 3. Generating interview questions and follow-up emails

[1164] The server uses a generative AI model (e.g., OpenAI's GPT, BERT) to generate customized interview questions based on the analysis results. The generated list of questions is stored in a database, and a notification is sent to the terminal. The terminal displays the received notification to the interviewer, who can then view the list of questions via a web interface or application.

[1165] After the interview, the server automatically generates a follow-up email and requests confirmation from the interviewer via the terminal. This email contains an expression of gratitude and information about the next steps.

[1166] For example, if a candidate has "Python development experience," "data analysis skills," and "project management experience," the server will generate interview questions like the following:

[1167] "What was the most challenging aspect of your project using Python?"

[1168] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1169] "What lessons have you learned from your project management experience?"

[1170] 4. Gathering feedback and updating the training program

[1171] After the interview, the user enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and an overall evaluation.

[1172] The server analyzes the collected feedback and stores it in a database as interviewer performance data. It then updates the training program for the next interview. This program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[1173] 5. Integration of the emotion engine and real-time feedback

[1174] The server uses an emotion engine to collect voice and facial expression data from interviewers and candidates in real time during interviews. The emotion engine estimates and analyzes emotional states based on voice tone, changes in facial expressions, and word choice.

[1175] The server sends the analysis results to the terminal in real time and provides feedback to the interviewer. Specific feedback may include comments such as, "The candidate appears nervous about this question."

[1176] Example of a prompt

[1177] You can use the following prompt statements to input data into the generative AI model and obtain the appropriate output.

[1178] "Analyze the next candidate's resume and generate customized interview questions."

[1179] "Please update the training program based on the interview feedback."

[1180] "Analyze the emotional data during the interview and provide real-time feedback."

[1181] In this way, the system can automate a series of processes, aiming to improve the quality of interviews and the offer acceptance rate.

[1182] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1183] Step 1:

[1184] Users upload candidate resumes and CVs using a web interface or a dedicated desktop application. The user selects the file and clicks the upload button. The input is the candidate's resume or CV, and the output is the uploaded file. Specifically, a file selection dialog opens, the user selects the file, and clicks the "Upload" button.

[1185] Step 2:

[1186] The terminal temporarily saves uploaded files to local storage. The input is the uploaded file, and the output is the file saved in local storage. Specifically, the system retrieves basic file information (filename, size, upload time, etc.) and saves the file to local storage.

[1187] Step 3:

[1188] The terminal transfers saved files to the server using a secure communication protocol (e.g., HTTPS). The input is a file stored in local storage, and the output is a file transferred to the server. Specifically, the terminal sends a secure HTTP request to the server and transfers the file data.

[1189] Step 4:

[1190] The server analyzes the received file using OCR technology. The input is the file uploaded to the server, and the output is the analyzed text data. Specifically, the server uses the Tesseract library to extract the text data from the file and saves it to a database.

[1191] Step 5:

[1192] The server analyzes text data extracted using natural language processing technology. The input is text data extracted by OCR, and the output is the identification of candidates' skill sets and career paths. Specifically, the server uses NLTK and spaCy to analyze the text data and identify important skill sets and work history information.

[1193] Step 6:

[1194] The server generates customized interview questions using a generative AI model based on the analysis results. The input is the analyzed candidate information, and the output is the generated list of interview questions. Specifically, the server inputs prompt sentences into the generative AI model (e.g., OpenAI's GPT) and generates the list of questions.

[1195] Step 7:

[1196] The server saves the generated question list to a database and sends a notification to the terminal. The input is the generated interview question list, and the output is the notification message. Specifically, the server saves the question list to the database and sends a notification message to the terminal.

[1197] Step 8:

[1198] The terminal displays notifications received from the server to the interviewer. The input is the notification message from the server, and the output is a display of the question list that the interviewer can view. Specifically, the terminal displays the notification message in the user interface, and clicking it displays the details of the question list.

[1199] Step 9:

[1200] The server automatically generates a follow-up email after the interview is completed. The input is the interview completion information, and the output is the follow-up email. Specifically, the server automatically generates the email content based on a follow-up email template and sends it to the terminal.

[1201] Step 10:

[1202] The terminal displays the follow-up email received from the server to the interviewer and requests their confirmation of its contents. The input is the received follow-up email, and the output is the follow-up email confirmed by the interviewer. Specifically, the terminal displays the contents of the follow-up email, allowing the interviewer to make corrections or approve it.

[1203] Step 11:

[1204] The user inputs interview feedback into the system interface. The input is the interviewer's feedback, and the output is feedback data. Specifically, the user enters the feedback into the input field and presses the submit button.

[1205] Step 12:

[1206] The server analyzes the collected feedback and updates the training program. The input is the collected feedback data, and the output is the updated training program. Specifically, the server analyzes the feedback data, generates the training content, and saves it to the database.

[1207] Step 13:

[1208] The server uses an emotion engine to collect and analyze emotional data during the interview. Inputs are voice and facial expression data, and output is the analyzed emotional data. Specifically, the server uses emotion recognition technology to analyze the emotional data in real time and provides it to the interviewer as feedback.

[1209] Step 14:

[1210] The server provides real-time feedback to the interviewer based on emotional data. The input is analyzed emotional data, and the output is a feedback message. Specifically, the server analyzes the emotional data and sends it to the interviewer's terminal so that they can receive visualized feedback.

[1211] Step 15:

[1212] The server customizes the training program based on emotional data. The input is emotional data, and the output is the customized training program. Specifically, the server analyzes the emotional data, optimizes the training content for the interviewer, and stores it in a database.

[1213] (Application Example 2)

[1214] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1215] This invention relates to a system for improving efficiency and accuracy in conventional recruitment processes and the work performance of factory robots. Conventional recruitment processes often rely heavily on the subjective opinions and experience of interviewers, resulting in lower offer acceptance rates. Similarly, in factory robot operations, delays in operator feedback can reduce efficiency and increase the likelihood of errors. This invention aims to solve these problems and improve the quality of interviews and factory work.

[1216] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1217] In this invention, the server includes means for analyzing candidate information and identifying the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, and means equipped with an emotion engine for collecting data in real time and providing feedback to the operator. This enables the provision of objective and customized questions to the interviewer, and improves work efficiency and accuracy by providing real-time feedback based on robotic and human emotion data.

[1218] A "candidate" is an individual who possesses the necessary information and skills to apply for a specific position.

[1219] "Means for analyzing information" refers to a device or program that has the function of identifying specific attributes or patterns based on input data and generating the necessary output based on those attributes or patterns.

[1220] A "skill set" refers to the abilities and experience required to perform a specific job or role.

[1221] A "career path" is a diagram or outline that shows an individual's work history and professional experience, illustrating the future progression of their career.

[1222] "Means for generating customized interview questions" refers to a device or program that automatically creates questions optimized for individual candidates based on analysis results.

[1223] A "follow-up email" is an email sent to a candidate after an interview, containing a thank you and information about the next steps.

[1224] An "interviewer" is an individual responsible for conducting interviews with candidates and evaluating their performance.

[1225] A "training program" is a curriculum of education and training designed to improve the skills and performance of interviewers.

[1226] "Means of collecting data in real time" refers to devices or programs that have the function of immediately acquiring and analyzing data generated on-site.

[1227] An "emotion engine" is a device or program that recognizes and analyzes emotions based on a person's facial expressions, tone of voice, and choice of words.

[1228] An "operator" is an individual responsible for operating machinery or equipment.

[1229] "Means of providing feedback" refers to a device or program that notifies operators or users in real time of suggestions for improvement or precautions based on analysis results.

[1230] 1. System Configuration

[1231] This invention provides a system for improving the work performance of factory robots. The system includes the following components:

[1232] 1. Means of entering candidate information

[1233] 2. A means of analyzing the entered candidate information to identify the candidate's skill set and career path.

[1234] 3. Means for generating customized interview questions based on analyzed information

[1235] 4. A method for generating follow-up emails to send after the interview has concluded.

[1236] 5. Means of collecting interviewer feedback and updating interviewer training programs.

[1237] 6. A system equipped with an emotion engine to collect data in real time and provide feedback to operators.

[1238] 7. A means of analyzing robot and human emotional data and providing real-time feedback to the operator based on the results.

[1239] 2. Program Processing

[1240] The program for realizing this invention performs the following processes:

[1241] Hardware and software usage

[1242] Camera: A video capture device is used to monitor the operation of factory robots. The camera captures the robot's movements as video in real time.

[1243] Emotion Recognition Model: Use an emotion recognition model trained with TensorFlow. Specifically, use the model named emotion_model.h5.

[1244] Face Recognition: Faces are detected using OpenCV's Haar Cascade Classifier. This allows for the analysis of the emotions of the operator and other participants during robot operation.

[1245] Natural Language Processing: Use the TextBlob library to analyze and process text data generated based on the analysis results.

[1246] Sending emails: Use the smtplib library to send notification emails to operators.

[1247] Specific example

[1248] As a concrete example, if a factory robot detects abnormal behavior during operation, the system will operate as follows:

[1249] First, cameras monitor the robot's movements within the factory and acquire video data in real time. The system detects the operator's facial expressions using a Haar cascade classifier and analyzes their emotions using an emotion recognition model (TensorFlow model).

[1250] For example, if an operator expresses emotions such as "surprise" or "anger," the system analyzes that emotion data and determines that a specific anomaly has occurred. In this case, a feedback message like the following is generated:

[1251] "An abnormality has been detected in the robot's operation. Please be careful."

[1252] This feedback message is displayed on the operator's screen in real time and is also notified via email. The smtplib library is used for sending emails.

[1253] Example of a prompt

[1254] Here are some specific examples of prompt statements to input into a generative AI model:

[1255] "Please prepare a detailed report on any issues related to the robot's operation. The report should include the type of anomaly detected, the countermeasures taken, and measures to prevent recurrence."

[1256] These components and processing procedures make it possible to improve the operational performance of factory robots and provide an efficient and safe working environment.

[1257] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1258] Step 1:

[1259] Capture images with a camera

[1260] The terminal acquires real-time video footage of robots and operators operating within the factory from cameras. The input is video data from the cameras, which is processed as a video stream. The output is raw video data.

[1261] Step 2:

[1262] Face detection

[1263] The server detects faces from the acquired video data using OpenCV's Haar cascade classifier. The input is the video data acquired in step 1, which is processed to extract specific areas (faces). The output is face coordinate (x, y, w, h) data.

[1264] Step 3:

[1265] Facial cropping and pre-processing

[1266] The device extracts the face portion from the video data based on the detected face coordinate data and converts it into a format that can be input into the emotion recognition model. The input consists of face coordinate data and raw video data, which are used to extract the face image, resize it, and convert it to grayscale. The output is a pre-processed face image.

[1267] Step 4:

[1268] Recognition of emotions

[1269] The server inputs pre-processed facial images into a TensorFlow emotion recognition model to analyze emotions. The input is pre-processed facial images, and the model predicts emotions. The output is emotion prediction data (labels such as anger, surprise, sadness, etc.).

[1270] Step 5:

[1271] Detection of abnormal operation

[1272] The server analyzes the output data of the emotion recognition model and recognizes abnormal emotions (e.g., anger) as abnormal behavior. The input is emotion prediction data, which is analyzed based on rules for determining whether an abnormality exists. The output is a flag indicating whether an abnormal behavior occurred.

[1273] Step 6:

[1274] Generating feedback messages

[1275] If an abnormal operation is detected, the server uses a generative AI model to generate a feedback message for the operator. The input is an abnormal operation flag and a prompt statement (e.g., "Please prepare a detailed report on the robot's operation issue"), and the AI ​​model generates an appropriate feedback message. The output is the feedback message text.

[1276] Step 7:

[1277] Real-time display of feedback

[1278] The terminal displays the generated feedback message on the operator's display in real time. The input is the feedback message text, and its content is transmitted to the operator via the display device. The output is visual feedback information.

[1279] Step 8:

[1280] Sending notification emails

[1281] The server generates a notification email containing a feedback message and sends it to the operator's email address. The input is the feedback message text and the operator's email address, and the email is sent using the smtplib library. The output is the sent notification email.

[1282] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1283] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1284] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1285] [Fourth Embodiment]

[1286] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1287] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1288] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1289] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1290] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1291] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1292] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1293] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1294] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1295] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1296] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1297] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1298] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1299] This invention relates to a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. This system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs. Specific embodiments for carrying out this invention are described below.

[1300] 1. System Overview

[1301] This system is equipped with programs to perform the following functions:

[1302] Method for entering candidate information

[1303] Means for analyzing the information of entered candidates

[1304] A means of generating customized interview questions based on analysis results.

[1305] A method for generating follow-up emails after an interview.

[1306] A means of collecting feedback from interviewers and updating the training program based on that feedback.

[1307] 2. Data Entry and Analysis

[1308] Users (interviewers or HR personnel) upload candidates' resumes and work histories to the system. File uploads are done via a web interface or a dedicated desktop application. The terminal temporarily stores these files in local storage and then transfers them to the server.

[1309] The server analyzes the received files using natural language processing techniques. Specifically, it uses OCR technology to extract text and identify important skill sets and career paths. The analyzed data is stored in a database.

[1310] 3. Generating Question and Follow-up Emails

[1311] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. This allows for the automatic creation of questions tailored to the candidate's specific skill set and career path. The generated list of questions is then provided to the interviewer via their terminal.

[1312] After the interview, the server generates a follow-up email, which is also sent to the interviewer via the terminal for confirmation. The follow-up email includes a thank-you message and notification of the next steps.

[1313] 4. Feedback and Training Program

[1314] After the interview is complete, the user (interviewer) enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations.

[1315] The server analyzes the interviewer's performance based on the collected feedback and updates the training program for the next interview. This training program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[1316] Specific example

[1317] For example, if a candidate's resume is uploaded and that candidate has "Python development experience," "data analysis skills," and "project management experience," the server will automatically generate interview questions like the following.

[1318] "What was the most challenging aspect of your project using Python?"

[1319] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1320] "What lessons have you learned from your project management experience?"

[1321] Furthermore, the follow-up email sent after the interview will be generated with content similar to the following.

[1322] "Thank you for participating in the interview. We will contact you regarding the next steps shortly."

[1323] In this way, the system can improve the quality of interviews and increase the offer acceptance rate.

[1324] The following describes the processing flow.

[1325] Step 1:

[1326] Users can upload candidates' resumes and work histories by dragging and dropping them into the system's upload form, or by selecting them from a file selection dialog.

[1327] Step 2:

[1328] The device temporarily saves the uploaded file to local storage and displays a progress bar to the user that says, "Sending data to the server."

[1329] Step 3:

[1330] The server receives the file sent from the terminal and stores it in the initial database. After that, it starts processing to convert the file contents into text format.

[1331] Step 4:

[1332] The server uses OCR (Optical Character Recognition) technology and libraries that extract text according to the file format to obtain the text content of the resume.

[1333] Step 5:

[1334] The server uses a natural language processing (NLP) model to analyze and extract important information such as skill sets, years of experience, educational background, and work history from the extracted text. The analysis results are stored in a database.

[1335] Step 6:

[1336] The terminal visualizes the analyzed data and displays a list of skill keywords and the candidate's career path on the screen. This allows interviewers to see which skills and experiences they should focus on evaluating.

[1337] Step 7:

[1338] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. The questions are generated to correspond to the candidate's specific skill set and career path.

[1339] Step 8:

[1340] The terminal displays a generated list of questions to the interviewer. The interviewer can select appropriate questions from the displayed list and edit them as needed.

[1341] Step 9:

[1342] The user conducts interviews with candidates using questions provided by the system. Data and observations obtained during the interview process are then entered into the system.

[1343] Step 10:

[1344] The server automatically generates follow-up emails to send after the interview using an AI model. These follow-up emails include expressions of gratitude and notifications for the next steps.

[1345] Step 11:

[1346] The device displays the contents of the follow-up email to the interviewer, informing them that they can review and revise it.

[1347] Step 12:

[1348] After the interview, the user (interviewer) uses a feedback form to evaluate the quality of the interview and the candidate's responses. The evaluation results are entered into the system.

[1349] Step 13:

[1350] The server receives feedback data and analyzes the interviewer's performance. Based on this analysis, it updates the training program for the next interview.

[1351] Step 14:

[1352] The device displays updated training programs and feedback results to the interviewer, providing information to help them prepare for the next interview.

[1353] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[1354] (Example 1)

[1355] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1356] In a company's recruitment process, it is crucial to streamline communication between interviewers and applicants and increase the offer acceptance rate. However, traditional manual interview preparation and follow-up email creation are time-consuming and labor-intensive, and suffer from inconsistencies in consistency and quality. Furthermore, insufficient interviewer training can lead to low interview quality. A system is needed to address these challenges and realize a more effective and efficient recruitment process.

[1357] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1358] In this invention, the server includes means for the user to input candidate information, means for the terminal to temporarily store the entered candidate information in local storage and transfer it to the server, means for the server to analyze the received candidate information and identify the candidate's skill set and career path, means for generating customized interview questions using a generative AI model based on the analyzed information, means for generating follow-up emails using the generative AI model after the interview is completed, and means for collecting interviewer feedback from the user and for the server to update the training program based on that feedback. This enables the system to streamline interview preparation, automate follow-up emails, and optimize interviewer training.

[1359] A "user" is a person or person who uses the system to input candidate information or interviewer feedback.

[1360] A "terminal" refers to a device such as a computer or tablet used by a user to temporarily store candidate information and transfer it to a server.

[1361] A "server" is the central hub of the entire system, a computer system that performs tasks such as analyzing candidate information, generating customized interview questions, creating follow-up emails, and updating training programs.

[1362] "Candidate information" refers to the content of resumes and work experience summaries obtained during the recruitment process.

[1363] "Local storage" refers to the area on a device where data is temporarily stored.

[1364] "Transfer" refers to the process of sending candidate information from a terminal to a server.

[1365] "Analysis" refers to the process of using natural language processing technology to analyze the received candidate information and identify their skill set and career path.

[1366] A "generative AI model" is an artificial intelligence model used to automatically generate customized interview questions and follow-up emails.

[1367] "Customized interview questions" are specific questions created by a generative AI model based on analyzed candidate information.

[1368] A "follow-up email" is an email sent to a candidate after an interview to express gratitude and provide information about the next steps.

[1369] "Feedback" refers to the opinions and comments that interviewers enter after an interview regarding the candidate's responses, the appropriateness of the questions, and other observations.

[1370] A "training program" is educational content that the server updates based on feedback to improve the interviewer's performance.

[1371] Modes for carrying out the invention

[1372] This invention is a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. The system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs.

[1373] Hardware and software to be used

[1374] hardware

[1375] Server: This is the central hub of the entire system, performing data analysis, generating questions using generative AI models, creating follow-up emails, and updating training programs.

[1376] Device: A device such as a computer or tablet used by the user (interviewer or HR representative).

[1377] software

[1378] Web Interface: Users upload candidate information via a web page using HTML / CSS / JavaScript.

[1379] OCR technology: Use Tesseract OCR to extract text from candidates' resumes and work histories.

[1380] Natural Language Processing (NLP) Libraries: Use Python-based NLP libraries (e.g., spaCy or NLTK) to analyze text and identify skill sets and career paths.

[1381] Generative AI Model: Uses OpenAI's GPT-3 and other technologies to generate customized interview questions and follow-up emails.

[1382] Database: We use MySQL or PostgreSQL to store analysis results and feedback data.

[1383] Specific operation of the system

[1384] 1. Data entry

[1385] Users upload resumes and work histories of interview candidates to the system. Uploads can be made via a web interface or a dedicated desktop application. For example, a user can select a file from their PC and press the upload button, and the file will be sent to the server.

[1386] 2. Data Analysis

[1387] The terminal temporarily saves uploaded files to local storage and then transfers them to the server. The server uses Tesseract OCR to extract text from the files and analyzes the text using Python's NLP library to identify important skill sets and career paths. The analyzed data is stored in a database and used for subsequent processing.

[1388] 3. Generating interview questions

[1389] Based on the analysis results, the server uses a generation AI model to generate customized interview questions. As a specific example, enter the following prompt text:

[1390] Skills extracted from the candidate's resume:

[1391] Python development experience

[1392] Data analysis skills

[1393] Project management experience

[1394] Based on the above skill set, generate questions that interviewers should ask candidates.

[1395] Based on this prompt, the generative AI model generates the following question.

[1396] 1. "What was the most challenging aspect of your project using Python?"

[1397] 2. "Please tell us about the specific tools and techniques you used when performing data analysis."

[1398] 3. "What lessons have you learned from your project management experience?"

[1399] 4. Conducting interviews and generating follow-up emails

[1400] The user conducts interviews with candidates using a list of questions provided through their device. After the interview, the server generates a follow-up email using a generative AI model. The follow-up email includes an expression of gratitude and a notification about the next steps. For example, it might contain the following:

[1401] Subject: Thank you for participating in the interview.

[1402] Main text:

[1403] [Candidate Name],

[1404] Thank you very much for participating in our interview. We will contact you regarding the next steps shortly. If you have any questions, please do not hesitate to contact us.

[1405] Thank you for your cooperation.

[1406] [Company Name] Recruitment Team

[1407] 5. Gathering feedback and updating the training program

[1408] After an interview, the user (interviewer) enters feedback through the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations. The server analyzes the collected feedback and evaluates the interviewer's performance. A training program for the next interview is automatically updated.

[1409] In this way, this system can improve the quality of interviews and increase the offer acceptance rate.

[1410] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1411] Step 1: User uploads candidate information.

[1412] Users upload resumes and work histories of interview candidates to the system. Input consists of candidate document files (PDF or Word files), primarily using a web interface or a dedicated desktop application. Specifically, users click a file selection button, select a file from their PC, and click the upload button. This action sends the file to the server.

[1413] Step 2: Saving files locally on the terminal and transferring them to the server.

[1414] The terminal temporarily saves the user-uploaded file to local storage. The file is then transferred to the specified server endpoint using an HTTP POST request. The input is the file uploaded by the user, and the output is the file transferred to the server.

[1415] Step 3: Server analyzes candidate information

[1416] The server analyzes the received file. First, it extracts text from the file using OCR technology (e.g., Tesseract OCR). This text extraction result is then analyzed using a Python NLP library (e.g., spaCy) to identify important skill sets and career paths. The input is the file transferred to the server, and the output is the analyzed information on the candidate's skill sets and career paths. Specifically, the server performs OCR processing and converts the resulting text data into structured data using natural language processing techniques.

[1417] Step 4: Server generates interview questions

[1418] The server generates customized interview questions using a generative AI model (e.g., OpenAI's GPT-3) based on the analysis results. The prompt text is input to the generative AI model, and the generated question list is obtained. The input is the analyzed candidate information and prompt text, and the output is a list of generated interview questions. The following prompt text is used as a concrete example.

[1419] Skills extracted from the candidate's resume:

[1420] Python development experience

[1421] Data analysis skills

[1422] Project management experience

[1423] Based on the above skill set, generate questions that interviewers should ask candidates.

[1424] The generative AI model generates a list of questions based on this.

[1425] Step 5: Conducting interviews with users

[1426] The user conducts interviews with candidates using a list of interview questions provided via the terminal. In this step, the user conducts actual interviews based on the generated question list and evaluates the candidates' responses. The input is the generated interview question list, and the output is the interview results.

[1427] Step 6: Server generates follow-up email

[1428] After the interview ends, the server uses a generative AI model to generate a follow-up email. The follow-up email includes an expression of gratitude and notification of the next steps. The input is the interview completion information, and the output is the generated follow-up email. As a specific example, the following follow-up email is generated:

[1429] Subject: Thank you for participating in the interview.

[1430] Main text:

[1431] [Candidate Name],

[1432] Thank you very much for participating in our interview. We will contact you regarding the next steps shortly. If you have any questions, please do not hesitate to contact us.

[1433] Thank you for your cooperation.

[1434] [Company Name] Recruitment Team

[1435] Step 7: User enters interview feedback.

[1436] After the interview, the user enters feedback through the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations. The input is interview feedback information, and the output is feedback data stored on the server.

[1437] Step 8: Server updates the training program

[1438] The server analyzes the collected feedback, evaluates the interviewer's performance, and updates the training program for the next interview. The input is feedback data, and the output is the updated training program. The server integrates this training program with other feedback data and interview data to provide optimal improvement suggestions.

[1439] (Application Example 1)

[1440] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1441] The goal is to improve the quality of customer service in physical stores and increase customer satisfaction. In particular, there is a need for a system that efficiently generates customized customer service questions based on customer interests and purchase history, automatically generates follow-up emails after customer service, and collects staff feedback and updates training programs.

[1442] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1443] In this invention, the server includes means for inputting customer information, means for analyzing the information to identify customer interests and purchase history, means for generating customized customer service questions, means for generating follow-up emails after the customer service is completed, and means for collecting staff feedback and updating training programs. This makes it possible to provide high-quality customer service tailored to the individual needs of customers and improve customer satisfaction.

[1444] A "customer" is a person who visits a commercial facility or service provider, and is the provider of information that identifies their interests and purchasing history.

[1445] "Information analysis" refers to the act of processing input data and extracting meaning and patterns from it, and in this invention, it is used to identify customer interests and purchase history.

[1446] "Customized customer service questions" are questions that are dynamically generated based on each customer's interests and purchase history, and are designed to provide individualized service.

[1447] A "follow-up email" is an email sent to a customer after the service has concluded, containing a message of gratitude and notification of the next steps.

[1448] "Staff" refers to employees who are engaged in providing services to customers in a physical store, and are the subjects of feedback and training programs.

[1449] "Feedback" refers to the opinions and evaluations that staff members have regarding customer service, and the training program is updated based on this feedback.

[1450] A "training program" refers to educational content aimed at improving staff customer service skills, and it is optimized based on feedback.

[1451] "Natural language processing technology" refers to a set of technologies used in the fields of computer science and artificial intelligence to understand and generate natural language, and is used in information analysis.

[1452] "Uploading data via the internet" refers to the act of transmitting customer information or other data to a system using an online network.

[1453] Modes for carrying out the invention

[1454] 1. System Program

[1455] Explain the program's processing in natural language.

[1456] This system is designed to provide customized customer service in physical stores. The specific processing flow is described below.

[1457] Customer information input and analysis

[1458] Users (staff) input basic customer information and purchase history into the system using devices such as smartphones and smart glasses. The devices transmit this information to the server via the internet. The server analyzes the received data using natural language processing techniques to identify customer interests and purchase history. This analysis uses natural language processing libraries such as Spacy and generative AI models such as GPT-3.

[1459] Generating customized customer service questions

[1460] The server generates prompts based on the analysis results, which use a generative AI model to generate customized customer service questions. The generated questions are then provided to the staff via the terminal. For example, for a customer with "Python development experience," "data analysis skills," and "project management experience," the following questions might be generated:

[1461] "What was the most challenging aspect of your project using Python?"

[1462] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1463] "What lessons have you learned from your project management experience?"

[1464] Generating follow-up emails

[1465] After the customer service session ends, the server automatically generates a follow-up email using a generation AI model. This email includes a message of gratitude and encourages the customer to visit again. The generated email is reviewed by staff via their terminals and then sent to the customer.

[1466] Example: "Thank you for visiting our store today. We look forward to your next visit."

[1467] Feedback collection and training program updates

[1468] Users (staff) input feedback about customer service using a terminal. This feedback includes customer reactions and the appropriateness of questions asked. The server stores this feedback information in a database and uses it to update staff training programs. This training program is optimized based on the collected feedback and reflected in future customer service interactions.

[1469] Specific example

[1470] Let's say a customer comes into the store and has "Python development experience," "data analysis skills," and "project management experience." The user (staff) enters the customer's basic information on their smartphone. The server analyzes this information and uses a generative AI model to generate customized questions like the following.

[1471] "What was the most challenging aspect of your project using Python?"

[1472] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1473] "What lessons have you learned from your project management experience?"

[1474] After serving a customer, the server automatically generates a follow-up email using a generative AI model, such as the following:

[1475] "Thank you for visiting our store today. We look forward to your next visit."

[1476] Example of a prompt

[1477] Prompt: "Generate customer service questions for a person with experience in Python development, data analysis, and project management."

[1478] In this way, this system can improve the quality of customer service in physical stores and increase customer satisfaction.

[1479] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1480] Step 1:

[1481] Users input basic customer information and purchase history using devices such as smartphones or smart glasses. Specifically, they enter data such as the customer's name, age, and past purchases into input fields. This data is transmitted from the device to the server via the internet. The input data consists of the customer's basic information and purchase history.

[1482] Step 2:

[1483] The server receives basic customer information and purchase history transmitted from the terminal. Next, natural language processing techniques (such as Spacy) are used to analyze this data and identify customer interests and purchasing patterns. As a result of the data analysis, the customer's areas of interest and purchasing tendencies are identified. The analyzed information includes the customer's interests and purchase history.

[1484] Step 3:

[1485] The server generates prompts based on the analysis results. For example, if "Python development experience," "data analysis skills," and "project management experience" are identified, the prompt might be "Generate customer service questions for a person with experience in Python development, data analysis, and project management." The generated prompts serve as input data for generating customized customer service questions.

[1486] Step 4:

[1487] The server uses a generative AI model (such as GPT-3) to take prompt text as input and generate customized customer service questions. Specific questions include, "What was the most challenging aspect of your project using Python?" The generated list of questions is sent to the terminal as customized customer service questions and provided to the staff. The prompt text is used as input data, and the generated customer service questions are used as output data.

[1488] Step 5:

[1489] The user (staff) interacts with the customer based on customized questions displayed on the terminal. Information obtained during the interaction can also be added to the input fields as needed. The entered data constitutes additional customer information.

[1490] Step 6:

[1491] After the customer service is complete, the server automatically generates a follow-up email using a generation AI model. For example, a follow-up email with content such as, "Thank you for visiting our store today. We look forward to your next visit," is generated. The generated follow-up email is sent to the customer's terminal as a follow-up message. The customer service results are used as input data, and the generated follow-up email is used as output data.

[1492] Step 7:

[1493] Users (staff) input feedback on customer service using a terminal. This feedback includes customer reactions and the appropriateness of questions asked. The input feedback data is sent to a server via the internet. The feedback serves as an evaluation of the staff's customer service and identifies areas for improvement.

[1494] Step 8:

[1495] The server receives feedback sent from the terminal and stores it in a database. It then updates the training program based on the collected feedback. For example, if feedback indicates that "the questions about Python were too difficult," "adjusting the difficulty of questions" will be added to the next training program. The updated training program is saved as instructional items for improving staff skills. The input data is the collected feedback, and the output data is the updated training program.

[1496] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1497] This invention relates to a system for improving communication between interviewers and applicants in a company's recruitment process and increasing the offer acceptance rate. This system includes various means for generating interview questions, providing follow-up emails, and updating interviewer training programs. Furthermore, by incorporating an emotion engine, it has the function of recognizing and analyzing the emotions of interviewers and candidates in real time. Specific embodiments for carrying out this invention are described below.

[1498] 1. System Overview

[1499] This system is equipped with programs to perform the following functions:

[1500] Method for entering candidate information

[1501] Means for analyzing the information of entered candidates

[1502] A means of generating customized interview questions based on analysis results.

[1503] A method for generating follow-up emails after an interview.

[1504] A means of collecting feedback from interviewers and updating the training program based on that feedback.

[1505] A means of combining an emotion engine that recognizes the emotions of interviewers and candidates.

[1506] A means of analyzing emotional data and providing real-time feedback to interviewers based on that data.

[1507] A means of customizing training programs based on emotional data

[1508] 2. Data Entry and Analysis

[1509] Users (interviewers or HR personnel) upload candidates' resumes and work histories to the system. File uploads are done via a web interface or a dedicated desktop application. The terminal temporarily stores these files in local storage and then transfers them to the server.

[1510] The server analyzes the received files using natural language processing techniques. Specifically, it uses OCR technology to extract text and identify important skill sets and career paths. The analyzed data is stored in a database.

[1511] 3. Generating Question and Follow-up Emails

[1512] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. This allows for the automatic creation of questions tailored to the candidate's specific skill set and career path. The generated list of questions is then provided to the interviewer via their terminal.

[1513] After the interview, the server generates a follow-up email, which is also sent to the interviewer via the terminal for confirmation. The follow-up email includes a thank-you message and notification of the next steps.

[1514] 4. Feedback and Training Program

[1515] After the interview is complete, the user (interviewer) enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and other observations.

[1516] The server analyzes the interviewer's performance based on the collected feedback and updates the training program for the next interview. This training program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[1517] 5. Integration of the Emotional Engine

[1518] The server uses an emotion engine that recognizes emotions from the voice and facial expressions of both the interviewer and the candidate during the interview. The emotion engine collects emotional data from voice tone, changes in facial expression, word choice, etc., and analyzes it in real time.

[1519] This emotional data is fed back to the interviewer in real time via the device, supporting smoother communication during the interview. Furthermore, the emotional data collected after the interview is used to customize the interviewer's training program.

[1520] Specific example

[1521] For example, if a candidate's resume is uploaded and that candidate has "Python development experience," "data analysis skills," and "project management experience," the server will automatically generate interview questions like the following.

[1522] "What was the most challenging aspect of your project using Python?"

[1523] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1524] "What lessons have you learned from your project management experience?"

[1525] During the interview, the server uses an emotion engine to analyze the emotions of both the interviewer and the candidate in real time, sending feedback to the terminal such as, "The candidate is nervous about this question." This allows the interviewer to adjust how they ask questions.

[1526] The follow-up email sent after the interview will contain content similar to the following.

[1527] "Thank you for participating in the interview. We will contact you regarding the next steps shortly."

[1528] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[1529] The following describes the processing flow.

[1530] Step 1:

[1531] Users upload candidates' resumes and work histories to the system. Users can drag and drop files or select them from a file selection dialog using a web interface or a dedicated desktop application.

[1532] Step 2:

[1533] The device temporarily saves the uploaded file to local storage and then transfers the file to the server. During this process, an upload progress bar is displayed to inform the user of the progress.

[1534] Step 3:

[1535] The server receives the file sent from the terminal and stores it in the initial database. Next, it starts the process of converting the file to a text format according to its format.

[1536] Step 4:

[1537] The server extracts the text content from the resume using OCR (Optical Character Recognition) technology and text extraction libraries tailored to the file format.

[1538] Step 5:

[1539] The server uses natural language processing (NLP) technology to analyze and extract important information such as skill sets, years of experience, educational background, and work history from the extracted text. The analysis results are stored in a database.

[1540] Step 6:

[1541] The terminal visualizes the analyzed data and displays a list of skill keywords and the candidate's career path on the screen. This allows interviewers to see which skills and experiences they should focus on evaluating.

[1542] Step 7:

[1543] Based on the analysis results, the server uses a generative AI model to generate customized interview questions. The questions are generated to correspond to the candidate's specific skill set and career path.

[1544] Step 8:

[1545] The terminal displays a generated list of questions to the interviewer. The interviewer can select appropriate questions from the displayed list and edit them as needed.

[1546] Step 9:

[1547] The server activates an emotion engine to recognize emotions from the voices and facial expressions of both the interviewer and the candidate during the interview. The emotion engine collects emotional data from voice tone, changes in facial expression, word choice, etc., and analyzes it in real time.

[1548] Step 10:

[1549] The device provides real-time feedback to the interviewer based on emotional data obtained from the emotion engine. This feedback includes specific advice such as, "The candidate is nervous," or "The interviewer should help the candidate relax more."

[1550] Step 11:

[1551] The user (interviewer) conducts interviews with candidates using questions provided by the system. Data and observations obtained during the interview process are then entered into the system.

[1552] Step 12:

[1553] The server automatically generates follow-up emails to send after the interview using an AI model. These follow-up emails include expressions of gratitude and notifications for the next steps.

[1554] Step 13:

[1555] The device displays the contents of the follow-up email to the interviewer, informing them that they can review and revise it.

[1556] Step 14:

[1557] After the interview, the user (interviewer) uses a feedback form to evaluate the quality of the interview and the candidate's responses. The evaluation results are entered into the system.

[1558] Step 15:

[1559] The server receives feedback data and analyzes the interviewer's performance. Based on this analysis, it updates the training program for the next interview.

[1560] Step 16:

[1561] The server analyzes the emotional data collected by the emotion engine and customizes the interviewer's training program based on that analysis. This allows interviewers to receive specific advice on how to improve their emotional recognition and communication skills.

[1562] Step 17:

[1563] The device displays updated training programs and feedback results to the interviewer, providing information to help them prepare for the next interview.

[1564] In this way, the system can automate a series of processes, improving the quality of interviews and the offer acceptance rate.

[1565] (Example 2)

[1566] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1567] Traditional recruitment processes lacked the ability to generate appropriate questions and provide real-time feedback to facilitate smooth communication between interviewers and applicants. Furthermore, sending follow-up emails after interviews and updating interviewer training programs were often done manually, which was inefficient. Additionally, the lack of automated recognition and feedback on emotions during interviews made it difficult to improve the quality of communication.

[1568] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1569] In this invention, the server includes means for inputting candidate information, means for analyzing the input candidate information and identifying the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, means for generating follow-up emails to be sent after the interview, means for collecting interviewer feedback and updating the interviewer's training program, means for combining an emotion engine that recognizes the emotions of the interviewer and candidate during the interview, means for analyzing the emotion data and providing real-time feedback to the interviewer based on it, and means for customizing the training program based on the emotion data. This makes it possible to improve the quality of communication between interviewers and applicants and to increase the efficiency and effectiveness of the entire recruitment process.

[1570] "Means of inputting candidate information" refers to the interface or device used by interviewers or HR personnel to input information such as candidates' resumes and work histories into a system.

[1571] "Means of analyzing input candidate information to identify candidates' skill sets and career paths" refers to the technologies and processes used by a system to analyze information provided by candidates using natural language processing techniques and extract important skills and work experience.

[1572] "Means for generating customized interview questions based on analyzed information" refers to technologies and processes that use a generation AI model to automatically create individually tailored interview questions based on the analysis results.

[1573] "Methods for generating follow-up emails to send after an interview" refers to the technology or process for automatically generating and sending emails containing expressions of gratitude and notifications regarding the next steps after an interview has concluded.

[1574] "Means of collecting interviewer feedback and updating interviewer training programs" refers to the techniques and processes for collecting feedback information from interviewers after interviews and using that information to continuously update training programs to improve interviewers' skills and knowledge.

[1575] "Means of combining an emotion engine to recognize the emotions of the interviewer and candidate during the interview" refers to a function that implements emotion recognition technology used to analyze emotions from voice, facial expressions, etc., during the interview.

[1576] "Means of analyzing emotional data and providing real-time feedback to interviewers based on that data" refers to technologies and processes that analyze collected emotional data, notify the interviewer of the analysis results in real time, and support the interviewer's response.

[1577] "Methods for customizing training programs based on emotional data" refers to techniques and processes for analyzing emotional data collected during interviews and using the results to customize and improve the interviewer's training program.

[1578] This invention is a system designed to improve communication between interviewers and applicants in a company's recruitment process, while simultaneously increasing the offer acceptance rate. This system functions by combining multiple software and hardware elements. Specific embodiments for carrying out this invention are described below.

[1579] 1. Data entry and temporary storage

[1580] Users upload candidate resumes and CVs to the system using a web interface or a dedicated desktop application. This involves selecting files and clicking an upload button. The terminal temporarily saves the uploaded files to local storage and then transfers them to the server.

[1581] The hardware used includes personal computers, tablets, and smartphones used by the user. The software includes web browsers and dedicated desktop applications.

[1582] 2. Transferring and analyzing data to the server

[1583] The device transfers saved files to the server using a secure communication protocol (e.g., HTTPS). The server extracts text data from the uploaded files using OCR (Optical Character Recognition) technology. This text data is then analyzed using natural language processing (NLP) technology to identify important skill sets and career paths.

[1584] Specific software includes libraries for using OCR technology (e.g., Tesseract) and frameworks using NLP technology (e.g., NLTK, spaCy). The analysis results are stored in a database.

[1585] 3. Generating interview questions and follow-up emails

[1586] The server uses a generative AI model (e.g., OpenAI's GPT, BERT) to generate customized interview questions based on the analysis results. The generated list of questions is stored in a database, and a notification is sent to the terminal. The terminal displays the received notification to the interviewer, who can then view the list of questions via a web interface or application.

[1587] After the interview, the server automatically generates a follow-up email and requests confirmation from the interviewer via the terminal. This email contains an expression of gratitude and information about the next steps.

[1588] For example, if a candidate has "Python development experience," "data analysis skills," and "project management experience," the server will generate interview questions like the following:

[1589] "What was the most challenging aspect of your project using Python?"

[1590] "Could you tell me about the specific tools and techniques you used when performing the data analysis?"

[1591] "What lessons have you learned from your project management experience?"

[1592] 4. Gathering feedback and updating the training program

[1593] After the interview, the user enters feedback into the system interface. This feedback includes the candidate's responses, the appropriateness of the questions, and an overall evaluation.

[1594] The server analyzes the collected feedback and stores it in a database as interviewer performance data. It then updates the training program for the next interview. This program is integrated with other interview databases and feedback data within the system to provide optimal improvement suggestions.

[1595] 5. Integration of the emotion engine and real-time feedback

[1596] The server uses an emotion engine to collect voice and facial expression data from interviewers and candidates in real time during interviews. The emotion engine estimates and analyzes emotional states based on voice tone, changes in facial expressions, and word choice.

[1597] The server sends the analysis results to the terminal in real time and provides feedback to the interviewer. Specific feedback may include comments such as, "The candidate appears nervous about this question."

[1598] Example of a prompt

[1599] You can use the following prompt statements to input data into the generative AI model and obtain the appropriate output.

[1600] "Analyze the next candidate's resume and generate customized interview questions."

[1601] "Please update the training program based on the interview feedback."

[1602] "Analyze the emotional data during the interview and provide real-time feedback."

[1603] In this way, the system can automate a series of processes, aiming to improve the quality of interviews and the offer acceptance rate.

[1604] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1605] Step 1:

[1606] Users upload candidate resumes and CVs using a web interface or a dedicated desktop application. The user selects the file and clicks the upload button. The input is the candidate's resume or CV, and the output is the uploaded file. Specifically, a file selection dialog opens, the user selects the file, and clicks the "Upload" button.

[1607] Step 2:

[1608] The terminal temporarily saves uploaded files to local storage. The input is the uploaded file, and the output is the file saved in local storage. Specifically, the system retrieves basic file information (filename, size, upload time, etc.) and saves the file to local storage.

[1609] Step 3:

[1610] The terminal transfers saved files to the server using a secure communication protocol (e.g., HTTPS). The input is a file stored in local storage, and the output is a file transferred to the server. Specifically, the terminal sends a secure HTTP request to the server and transfers the file data.

[1611] Step 4:

[1612] The server analyzes the received file using OCR technology. The input is the file uploaded to the server, and the output is the analyzed text data. Specifically, the server uses the Tesseract library to extract the text data from the file and saves it to a database.

[1613] Step 5:

[1614] The server analyzes text data extracted using natural language processing technology. The input is text data extracted by OCR, and the output is the identification of candidates' skill sets and career paths. Specifically, the server uses NLTK and spaCy to analyze the text data and identify important skill sets and work history information.

[1615] Step 6:

[1616] The server generates customized interview questions using a generative AI model based on the analysis results. The input is the analyzed candidate information, and the output is the generated list of interview questions. Specifically, the server inputs prompt sentences into the generative AI model (e.g., OpenAI's GPT) and generates the list of questions.

[1617] Step 7:

[1618] The server saves the generated question list to a database and sends a notification to the terminal. The input is the generated interview question list, and the output is the notification message. Specifically, the server saves the question list to the database and sends a notification message to the terminal.

[1619] Step 8:

[1620] The terminal displays notifications received from the server to the interviewer. The input is the notification message from the server, and the output is a display of the question list that the interviewer can view. Specifically, the terminal displays the notification message in the user interface, and clicking it displays the details of the question list.

[1621] Step 9:

[1622] The server automatically generates a follow-up email after the interview is completed. The input is the interview completion information, and the output is the follow-up email. Specifically, the server automatically generates the email content based on a follow-up email template and sends it to the terminal.

[1623] Step 10:

[1624] The terminal displays the follow-up email received from the server to the interviewer and requests their confirmation of its contents. The input is the received follow-up email, and the output is the follow-up email confirmed by the interviewer. Specifically, the terminal displays the contents of the follow-up email, allowing the interviewer to make corrections or approve it.

[1625] Step 11:

[1626] The user inputs interview feedback into the system interface. The input is the interviewer's feedback, and the output is feedback data. Specifically, the user enters the feedback into the input field and presses the submit button.

[1627] Step 12:

[1628] The server analyzes the collected feedback and updates the training program. The input is the collected feedback data, and the output is the updated training program. Specifically, the server analyzes the feedback data, generates the training content, and saves it to the database.

[1629] Step 13:

[1630] The server uses an emotion engine to collect and analyze emotional data during the interview. Inputs are voice and facial expression data, and output is the analyzed emotional data. Specifically, the server uses emotion recognition technology to analyze the emotional data in real time and provides it to the interviewer as feedback.

[1631] Step 14:

[1632] The server provides real-time feedback to the interviewer based on emotional data. The input is analyzed emotional data, and the output is a feedback message. Specifically, the server analyzes the emotional data and sends it to the interviewer's terminal so that they can receive visualized feedback.

[1633] Step 15:

[1634] The server customizes the training program based on emotional data. The input is emotional data, and the output is the customized training program. Specifically, the server analyzes the emotional data, optimizes the training content for the interviewer, and stores it in a database.

[1635] (Application Example 2)

[1636] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1637] This invention relates to a system for improving efficiency and accuracy in conventional recruitment processes and the work performance of factory robots. Conventional recruitment processes often rely heavily on the subjective opinions and experience of interviewers, resulting in lower offer acceptance rates. Similarly, in factory robot operations, delays in operator feedback can reduce efficiency and increase the likelihood of errors. This invention aims to solve these problems and improve the quality of interviews and factory work.

[1638] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1639] In this invention, the server includes means for analyzing candidate information and identifying the candidate's skill set and career path, means for generating customized interview questions based on the analyzed information, and means equipped with an emotion engine for collecting data in real time and providing feedback to the operator. This enables the provision of objective and customized questions to the interviewer, and improves work efficiency and accuracy by providing real-time feedback based on robotic and human emotion data.

[1640] A "candidate" is an individual who possesses the necessary information and skills to apply for a specific position.

[1641] "Means for analyzing information" refers to a device or program that has the function of identifying specific attributes or patterns based on input data and generating the necessary output based on those attributes or patterns.

[1642] A "skill set" refers to the abilities and experience required to perform a specific job or role.

[1643] A "career path" is a diagram or outline that shows an individual's work history and professional experience, illustrating the future progression of their career.

[1644] "Means for generating customized interview questions" refers to a device or program that automatically creates questions optimized for individual candidates based on analysis results.

[1645] A "follow-up email" is an email sent to a candidate after an interview, containing a thank you and information about the next steps.

[1646] An "interviewer" is an individual responsible for conducting interviews with candidates and evaluating their performance.

[1647] A "training program" is a curriculum of education and training designed to improve the skills and performance of interviewers.

[1648] "Means of collecting data in real time" refers to devices or programs that have the function of immediately acquiring and analyzing data generated on-site.

[1649] An "emotion engine" is a device or program that recognizes and analyzes emotions based on a person's facial expressions, tone of voice, and choice of words.

[1650] An "operator" is an individual responsible for operating machinery or equipment.

[1651] "Means of providing feedback" refers to a device or program that notifies operators or users in real time of suggestions for improvement or precautions based on analysis results.

[1652] 1. System Configuration

[1653] This invention provides a system for improving the work performance of factory robots. The system includes the following components:

[1654] 1. Means of entering candidate information

[1655] 2. A means of analyzing the entered candidate information to identify the candidate's skill set and career path.

[1656] 3. Means for generating customized interview questions based on analyzed information

[1657] 4. A method for generating follow-up emails to send after the interview has concluded.

[1658] 5. Means of collecting interviewer feedback and updating interviewer training programs.

[1659] 6. A system equipped with an emotion engine to collect data in real time and provide feedback to operators.

[1660] 7. A means of analyzing robot and human emotional data and providing real-time feedback to the operator based on the results.

[1661] 2. Program Processing

[1662] The program for realizing this invention performs the following processes:

[1663] Hardware and software usage

[1664] Camera: A video capture device is used to monitor the operation of factory robots. The camera captures the robot's movements as video in real time.

[1665] Emotion Recognition Model: Use an emotion recognition model trained with TensorFlow. Specifically, use the model named emotion_model.h5.

[1666] Face Recognition: Faces are detected using OpenCV's Haar Cascade Classifier. This allows for the analysis of the emotions of the operator and other participants during robot operation.

[1667] Natural Language Processing: Use the TextBlob library to analyze and process text data generated based on the analysis results.

[1668] Sending emails: Use the smtplib library to send notification emails to operators.

[1669] Specific example

[1670] As a concrete example, if a factory robot detects abnormal behavior during operation, the system will operate as follows:

[1671] First, cameras monitor the robot's movements within the factory and acquire video data in real time. The system detects the operator's facial expressions using a Haar cascade classifier and analyzes their emotions using an emotion recognition model (TensorFlow model).

[1672] For example, if an operator expresses emotions such as "surprise" or "anger," the system analyzes that emotion data and determines that a specific anomaly has occurred. In this case, a feedback message like the following is generated:

[1673] "An abnormality has been detected in the robot's operation. Please be careful."

[1674] This feedback message is displayed on the operator's screen in real time and is also notified via email. The smtplib library is used for sending emails.

[1675] Example of a prompt

[1676] Here are some specific examples of prompt statements to input into a generative AI model:

[1677] "Please prepare a detailed report on any issues related to the robot's operation. The report should include the type of anomaly detected, the countermeasures taken, and measures to prevent recurrence."

[1678] These components and processing procedures make it possible to improve the operational performance of factory robots and provide an efficient and safe working environment.

[1679] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1680] Step 1:

[1681] Capture images with a camera

[1682] The terminal acquires real-time video footage of robots and operators operating within the factory from cameras. The input is video data from the cameras, which is processed as a video stream. The output is raw video data.

[1683] Step 2:

[1684] Face detection

[1685] The server detects faces from the acquired video data using OpenCV's Haar cascade classifier. The input is the video data acquired in step 1, which is processed to extract specific areas (faces). The output is face coordinate (x, y, w, h) data.

[1686] Step 3:

[1687] Facial cropping and pre-processing

[1688] The device extracts the face portion from the video data based on the detected face coordinate data and converts it into a format that can be input into the emotion recognition model. The input consists of face coordinate data and raw video data, which are used to extract the face image, resize it, and convert it to grayscale. The output is a pre-processed face image.

[1689] Step 4:

[1690] Recognition of emotions

[1691] The server inputs pre-processed facial images into a TensorFlow emotion recognition model to analyze emotions. The input is pre-processed facial images, and the model predicts emotions. The output is emotion prediction data (labels such as anger, surprise, sadness, etc.).

[1692] Step 5:

[1693] Detection of abnormal operation

[1694] The server analyzes the output data of the emotion recognition model and recognizes abnormal emotions (e.g., anger) as abnormal behavior. The input is emotion prediction data, which is analyzed based on rules for determining whether an abnormality exists. The output is a flag indicating whether an abnormal behavior occurred.

[1695] Step 6:

[1696] Generating feedback messages

[1697] If an abnormal operation is detected, the server uses a generative AI model to generate a feedback message for the operator. The input is an abnormal operation flag and a prompt statement (e.g., "Please prepare a detailed report on the robot's operation issue"), and the AI ​​model generates an appropriate feedback message. The output is the feedback message text.

[1698] Step 7:

[1699] Real-time display of feedback

[1700] The terminal displays the generated feedback message on the operator's display in real time. The input is the feedback message text, and its content is transmitted to the operator via the display device. The output is visual feedback information.

[1701] Step 8:

[1702] Sending notification emails

[1703] The server generates a notification email containing a feedback message and sends it to the operator's email address. The input is the feedback message text and the operator's email address, and the email is sent using the smtplib library. The output is the sent notification email.

[1704] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1705] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1706] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1707] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1708] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1709] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1710] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1711] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1712] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1713] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1714] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1715] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1716] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1717] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1718] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1719] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1720] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1721] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1722] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1723] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1724] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1725] The following is further disclosed regarding the embodiments described above.

[1726] (Claim 1)

[1727] A means of entering candidate information,

[1728] A means of analyzing the entered candidate information to identify the candidate's skill set and career path,

[1729] A means for generating customized interview questions based on analyzed information,

[1730] A method for generating a follow-up email to send after the interview,

[1731] A means of collecting feedback from interviewers and updating the interviewer training program,

[1732] A system that includes this.

[1733] (Claim 2)

[1734] The system according to claim 1, which uses natural language processing techniques to analyze candidate information.

[1735] (Claim 3)

[1736] The system according to claim 1, comprising means for uploading a file via the internet to input candidate information.

[1737] (Claim 4)

[1738] The system according to claim 1, comprising means for providing an interface for an interviewer to input feedback.

[1739] (Claim 5)

[1740] The system according to claim 1, comprising means for visualizing the analyzed candidate information and displaying it to the interviewer.

[1741] "Example 1"

[1742] (Claim 1)

[1743] A means for users to input candidate information,

[1744] A means for the terminal to temporarily store the entered candidate information in local storage and transfer it to the server,

[1745] The server analyzes the candidate information it receives and identifies the candidate's skill set and career path.

[1746] A means for generating customized interview questions using a generative AI model based on analyzed information,

[1747] A method for generating follow-up emails using a generation AI model after the interview,

[1748] A means for collecting feedback from users to interviewers and for the server to update the training program based on that feedback,

[1749] A system that includes this.

[1750] (Claim 2)

[1751] The system according to claim 1, wherein the server uses natural language processing techniques to analyze candidate information.

[1752] (Claim 3)

[1753] The system according to claim 1, comprising means for a user to upload a file via the internet to input candidate information.

[1754] "Application Example 1"

[1755] (Claim 1)

[1756] Means of entering customer information,

[1757] A means of analyzing entered customer information to identify customer interests and purchase history,

[1758] A means for generating customized customer service questions based on analyzed information,

[1759] A method for generating follow-up emails to send after customer service has ended,

[1760] A means of collecting staff feedback and updating staff training programs,

[1761] A system that includes this.

[1762] (Claim 2)

[1763] The system according to claim 1, which uses natural language processing technology to analyze customer information.

[1764] (Claim 3)

[1765] The system according to claim 1, comprising means for uploading data via the internet to input customer information.

[1766] "Example 2 of combining an emotion engine"

[1767] (Claim 1)

[1768] A means of entering candidate information,

[1769] A means of analyzing the entered candidate information to identify the candidate's skill set and career path,

[1770] A means for generating customized interview questions based on analyzed information,

[1771] A method for generating a follow-up email to send after the interview,

[1772] A means of collecting feedback from interviewers and updating the interviewer training program,

[1773] A means of combining an emotion engine that recognizes the emotions of the interviewer and the candidate during the interview,

[1774] A means of analyzing emotional data and providing real-time feedback to interviewers based on that data,

[1775] A means of customizing training programs based on emotional data,

[1776] A system that includes this.

[1777] (Claim 2)

[1778] The system according to claim 1, which uses natural language processing techniques to analyze candidate information.

[1779] (Claim 3)

[1780] The system according to claim 1, comprising means for uploading a file via ...

Claims

1. A means of entering candidate information, A means of analyzing the entered candidate information to identify the candidate's skill set and career path, A means for generating customized interview questions based on analyzed information, A method for generating a follow-up email to send after the interview, A means of collecting feedback from interviewers and updating the interviewer training program, A system that includes this.

2. The system according to claim 1, which uses natural language processing technology to analyze candidate information.

3. The system according to claim 1, comprising means for uploading a file via the internet to input candidate information.

4. The system according to claim 1, comprising means for providing an interface for an interviewer to input feedback.

5. The system according to claim 1, comprising means for visualizing the analyzed candidate information and displaying it to the interviewer.

Images