System

Abstract

A system is provided.SOLUTION: A system, comprising: means for generating a customized virtual interviewer based on a user's request; means for performing real-time interaction with a candidate and conducting question-and-answer sessions; means for collecting and analyzing interview data and providing an optimized interview experience; and means for automatically learning a questioning style and an evaluation standard of a virtual interviewer based on user feedback and interview results.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many companies often lose recruitment opportunities due to a lack of interviewer skills and resources. Furthermore, human errors, such as stressful interviews or gaffes by interviewers, can damage a company's image. These challenges make it difficult for companies to find the right talent quickly, reducing the efficiency and effectiveness of the recruitment process. [Means for solving the problem]

[0005] The present invention provides a system that utilizes a generative AI model to generate a customized virtual interviewer based on user requirements. The system has a means for conducting real-time dialogue with candidates and conducting Q&A sessions. It also has a means for collecting and analyzing interview data to provide an optimized interview experience. It also includes a means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results. This enables companies to efficiently and effectively evolve their interview processes and quickly find top talent.

[0006] A "user" is someone who uses the system to generate virtual interviewers and to optimize a company's recruitment process.

[0007] "Requirements" refer to conditions such as industry, job type, and evaluation skills that users specify to the system, as well as expected functions.

[0008] "Customization" refers to modifying or adjusting system settings or profiles at the user's request to suit their specific needs.

[0009] A "virtual interviewer" is an AI-based interviewer that can conduct realistic dialogue and question-and-answer sessions based on a generative AI model.

[0010] A "generative AI model" is an algorithm or program that uses deep learning and natural language processing techniques to generate a virtual interviewer based on specific conditions and requirements.

[0011] "Real-time dialogue" refers to a dialogue format in which a virtual interviewer immediately asks questions and answers with the candidate, providing immediate responses.

[0012] "Q&A" is a process in which a virtual interviewer asks a candidate questions and determines the next question based on the candidate's answers.

[0013] "Interview data" is a general term for information such as candidate responses, evaluation results, and feedback generated during the interview process.

[0014] "Analysis" is the process of analyzing the collected interview data using statistical and machine learning methods to derive specific insights and conclusions.

[0015] "Optimization" refers to adjusting the means and processes to maximize the functionality and performance of a system based on user requirements and feedback.

[0016] "Feedback" refers to information about improvements and evaluations provided by users after the interview is completed, and this information is used to improve the system.

[0017] "Questioning style" refers to the way or pattern of questions a virtual interviewer uses with a candidate.

[0018] "Evaluation criteria" are the standards or criteria used to evaluate a candidate's responses and thereby determine the candidate's abilities and aptitude.

[0019] These definitions are provided to aid in the understanding of important technical terms in the present invention. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention will be described below as a specific method for implementing the invention, with reference to the operation of a system using a user, a terminal, and a server. The invention is implemented based on the following configuration.

[0042] 1. Interviewer generation and customization

[0043] 1-1: Receive customization requests

[0044] Users use the system to submit customization requests to generate interviewers for specific industries and job types. For example, if the user is looking to hire a software engineer, the user inputs the skills (e.g., programming skills and algorithm knowledge) and characteristics (e.g., problem-solving ability and communication ability) they want to evaluate on the device's operation screen. The device then collects this information and sends it to the server as a customization request.

[0045] 1-2: Generate interviewers

[0046] The server analyzes the customization request sent by the user and generates a virtual interviewer using a generative AI model based on the analysis. The virtual interviewer has question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0047] 2. Real-time Q&A

[0048] 2-1: Start of the interview

[0049] The user sends an interview invitation to the candidate and schedules the interview. The terminal is prepared so that the candidate can log in to the system at the specified date and time. The server activates the virtual interviewer at the start time of the interview.

[0050] 2-2: Providing questions and recording responses

[0051] The server asks the candidate an initial question via a virtual interviewer. For example, "Tell me about your programming experience." When the candidate enters their response via their terminal, it is sent to the server. The server analyzes the candidate's response and generates the next question in real time based on the content. For example, if the candidate answers "I have experience with Python and Java," the server generates the next question in real time: "In what specific projects have you used those languages?" This allows for a detailed evaluation of the candidate's skills and aptitude.

[0052] 3. Evolving learning functions

[0053] 3-1: Collecting and sending feedback

[0054] After the interview, the user enters feedback through the system, including an evaluation of the overall interview and evaluation of specific questions and responses. The terminal collects this feedback and sends it to the server.

[0055] 3-2: Optimizing question style and evaluation criteria

[0056] The server analyzes the collected feedback and interview performance data. For example, if feedback indicates that there are not enough questions to evaluate the candidate's logical thinking, the system can use this information to adjust the questioning style and add a question such as "Please explain the specific steps you took to solve the problem" to the next interview. In this way, the virtual interviewer can continuously evolve and provide more effective interviews.

[0057] 4. Interview data analysis

[0058] 4-1: Collection of interview data

[0059] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback.

[0060] 4-2: Analyzing data and providing insights

[0061] The server analyzes the collected data and provides the user with insights into the interview process, such as analyzing the average technical skill scores of multiple candidates or specific skill deficiencies, and generates a report that is provided to the user, allowing the user to gain specific insights to further optimize the hiring process.

[0062] 5. Providing flexible and scalable solutions

[0063] 5-1: Scaling up

[0064] The server dynamically adjusts system resources according to the company's hiring volume. For example, if a large number of interviews are being conducted at once, the necessary computing resources will be automatically increased. The server also distributes the load to ensure stable operation of terminals.

[0065] 5-2: Meeting different hiring needs

[0066] Users can use their devices to customize settings for different job types and departments. For example, they can set up the system to generate virtual interviewers that are optimal for each job type, so that it can accommodate not only software engineers but also marketing and sales positions. The server generates the appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[0067] The above is an embodiment of the present invention, which enables companies to recruit talented personnel efficiently and effectively.

[0068] The processing flow will be explained below.

[0069] 1. Interviewer generation and customization

[0070] Step 1:

[0071] The user inputs the industry, job type, and skills and characteristics they want to evaluate through the device's operation screen.

[0072] Step 2:

[0073] The terminal collects input information from the user and transmits it to the server as a customization request.

[0074] Step 3:

[0075] The server analyzes the received request and uses a generative AI model to generate a virtual interviewer profile, which includes question patterns and evaluation criteria appropriate for the industry and job type.

[0076] Step 4:

[0077] The server stores the generated virtual interviewer profile and the interview is ready.

[0078] 2. Real-time Q&A

[0079] Step 1:

[0080] The user sends interview invitations to candidates and schedules interviews.

[0081] Step 2:

[0082] The terminal will be ready for the candidate to log into the system at the appointed date and time.

[0083] Step 3:

[0084] The server activates the virtual interviewer at the start time of the interview.

[0085] Step 4:

[0086] The server asks the candidate an initial question via a virtual interviewer.

[0087] Step 5:

[0088] The candidate enters their response through a terminal.

[0089] Step 6:

[0090] The terminal sends the candidate's response to the server.

[0091] Step 7:

[0092] The server analyzes the candidate's responses and generates subsequent questions in real time based on their content. For example, if a question about technical skills follows, the server generates a question such as, "What specific projects have you used those languages ​​in?"

[0093] Step 8:

[0094] The server generates the next question and sends it to the terminal, which then presents it to the candidate.

[0095] Step 9:

[0096] This process is repeated until all questions have been completed.

[0097] 3. Evolving learning functions

[0098] Step 1:

[0099] After the interview, the user enters feedback through the system.

[0100] Step 2:

[0101] The device collects the feedback and sends it to the server.

[0102] Step 3:

[0103] The server analyzes the collected feedback and interview performance data. Based on the analysis results, the question style and evaluation criteria are automatically adjusted according to the AI ​​model. For example, if feedback indicates that there are a lack of questions that evaluate logical thinking, relevant question patterns are added.

[0104] 4. Interview data analysis

[0105] Step 1:

[0106] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback.

[0107] Step 2:

[0108] The server analyzes the collected data and provides the user with insight into the interview process, such as average technical skill scores or identifying skill gaps, and generates a report of the results.

[0109] Step 3:

[0110] The terminal displays the analysis results provided by the server to the user.

[0111] 5. Providing flexible and scalable solutions

[0112] Step 1:

[0113] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the computing resources will be automatically increased.

[0114] Step 2:

[0115] Terminals distribute the load to ensure stable operation of the system.

[0116] Step 3:

[0117] Users can customize settings for different job types or departments from their terminals.

[0118] Step 4:

[0119] The server generates an appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[0120] The above are the specific processing steps of the present invention, and this flow enables companies to recruit talented personnel efficiently and effectively.

[0121] Example 1

[0122] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0123] In the modern recruitment process, there are many challenges in candidate evaluation and interview efficiency. In particular, evaluations are often influenced by the interviewer's subjectivity, preparing for and conducting interviews requires a great deal of time and resources, and continuous improvement based on feedback is difficult. There is a need to solve these challenges and provide a more effective and efficient interview process.

[0124] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0125] In this invention, the server includes a means for generating a customized virtual interviewer based on user requests, a means for conducting real-time dialogue with candidates and conducting Q&A sessions, and a means for collecting and analyzing interview data to provide an optimized interview experience. This streamlines the interview process and enables fair and objective candidate evaluation. Furthermore, the virtual interviewer's questioning style and evaluation criteria can be automatically learned using user feedback and interview results, enabling continuous improvement. Furthermore, because the customized virtual interviewer is generated using a generative AI model, it is easy to create a virtual interviewer with question patterns and evaluation criteria suited to specific industries and job types.

[0126] A "user request" is a request by a user to customize the virtual interviewer based on a particular industry, job type, and assessment skills when generating the virtual interviewer.

[0127] A "customized virtual interviewer" is a virtual interviewer with question patterns and evaluation criteria suited to the needs of a specific industry or job type, according to the user's request.

[0128] A "generative AI model" is an artificial intelligence model that analyzes user requirements and generates a customized virtual interviewer based on those requirements.

[0129] "Real-time dialogue" refers to the process of an interview in which the candidate and virtual interviewer interactively exchange questions and answers, with immediate responses and feedback.

[0130] A "question and answer" is a series of interactions in which a virtual interviewer asks a candidate questions and the candidate responds.

[0131] "Interview data" refers to data collected during an interview, including information such as the candidate's responses, evaluation scores, and feedback.

[0132] An "optimized interview experience" refers to an interview process that improves the quality and efficiency of interviews by analyzing interview data and utilizing feedback.

[0133] "Feedback" refers to the evaluation of the entire interview provided by the user after the interview is completed, as well as evaluation comments on specific questions and answers.

[0134] "Questioning style" is a concept that refers to the format and manner in which a virtual interviewer asks questions to a candidate.

[0135] "Evaluation criteria" are indicators or standards that the virtual interviewer uses to evaluate the candidate's responses, and are set based on the user's requests.

[0136] A "profile" refers to a generated set of information about a virtual interviewer, including data related to interview preparation.

[0137] "Analysis" is the process of analyzing the collected interview data and generating insights and recommendations based on the results.

[0138] The present invention is an interview support system implemented using a user, a terminal, and a server, and a specific implementation method thereof will be described below. Implementing the invention requires cooperation between hardware and software.

[0139] 1. Interviewer generation and customization

[0140] To generate a virtual interviewer for a specific industry and job type, the user inputs customization requests using the device's operation screen. Examples of input include evaluation items such as "programming skills," "algorithm knowledge," "problem-solving ability," and "communication ability." The device collects this input information and sends it to the server as a customization request.

[0141] The server analyzes the received customization request and uses a generative AI model to generate a customized virtual interviewer with question patterns and evaluation criteria suited to the specific industry and job type. The generated virtual interviewer profile is stored on the server and is ready for the interview.

[0142] 2. Real-time Q&A

[0143] The user sends an interview invitation to the candidate from the terminal and arranges the interview date. The terminal makes the necessary preparations so that the candidate can log in to the system at the specified date and time. The server activates the virtual interviewer at the start time of the interview.

[0144] When the interview begins, the server provides the candidate with the first question through the virtual interviewer. A specific example question is, "Tell me about your programming experience." The candidate enters their response through their terminal, which is then sent to the server. The server analyzes the candidate's response and generates the next question in real time based on the content. For example, if the candidate answers, "I have experience with Python and Java," the server generates a question such as, "In what specific projects have you used those languages?"

[0145] 3. Evolving learning functions

[0146] After the interview, the user enters feedback through the system. The feedback includes an evaluation of the entire interview and evaluation of specific questions and answers. The terminal collects this feedback and sends it to the server.

[0147] The server analyzes the collected feedback and interview performance data to optimize the virtual interviewer's questioning style and evaluation criteria. For example, if feedback indicates that there are not enough questions to evaluate the candidate's logical thinking, the server will add a question such as "Please explain the specific steps you took to solve the problem" to the next interview. In this way, the virtual interviewer continuously evolves and provides more effective interviews.

[0148] 4. Interview data analysis

[0149] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback. The server analyzes the collected data and provides users with insights into the interview process. For example, the average technical skill scores of multiple candidates and specific skill deficiencies are analyzed and provided to users as a report. This allows users to gain specific insights to further optimize the hiring process.

[0150] 5. Providing flexible and scalable solutions

[0151] The server dynamically adjusts system resources according to the company's hiring volume. For example, when conducting a large number of interviews at once, it automatically increases the necessary computing resources and distributes the load to ensure stable terminal operation. Users can also use their terminals to customize settings for different job types and departments. For example, it is possible to set up the system to generate virtual interviewers optimized for each job type, so that it can accommodate not only software engineers but also marketing and sales positions. The server generates appropriate virtual interviewers based on each request and supports the smooth conduct of interviews.

[0152] Specific examples

[0153] An example prompt might be, "What skills are necessary to generate software engineer interviewers?", to which the system generates specific answers such as "programming skills, knowledge of algorithms, problem-solving ability, and communication skills."

[0154] By following the above procedure, the present invention can be implemented, enabling companies to recruit talented personnel efficiently and effectively.

[0155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0156] Step 1: Enter your customization requests

[0157] The user inputs a request for customizing the virtual interviewer using the operation screen of the terminal.

[0158] As an example of input, select evaluation items such as "programming skills" and "knowledge of algorithms."

[0159] The terminal collects the input customization requests and transmits them to the server as a customization request.

[0160] Input: The customization request that the user types into the terminal.

[0161] Output: Customization request data generated by the device

[0162] Step 2: Parsing the customization request

[0163] The server receives a customization request sent from the terminal.

[0164] The server analyzes the received data and extracts parameters to input into the generative AI model.

[0165] Data analysis involves classifying and organizing information based on industry, job type, and assessed skills.

[0166] Input: Customization request sent from the device

[0167] Output: Analysis results to be input into the generative AI model

[0168] Step 3: Generate a virtual interviewer

[0169] The server generates a virtual interviewer using a generative AI model based on the analysis results.

[0170] The generated virtual interviewer has a specific question pattern and evaluation criteria.

[0171] The generated virtual interviewer profile is stored in the server.

[0172] Input: Analysis results (parameters reflecting user customization requests)

[0173] Output: Virtual interviewer profile

[0174] Step 4: Schedule and prepare for the interview

[0175] The user operates the terminal to set an interview date in order to send an interview invitation to the candidate.

[0176] The terminal transmits the set schedule information to the server and prepares for the candidate to log in at the specified date and time.

[0177] The server activates the virtual interviewer at the start time of the interview.

[0178] Input: Interview schedule information

[0179] Output: Sending interview invitations and preparing the system

[0180] Step 5: Start the interview and ask questions

[0181] The server provides the candidate with initial questions through a virtual interviewer.

[0182] The server generates questions based on the evaluation items set by the user.

[0183] The candidate enters their response through the terminal, which is then sent to the server.

[0184] Input: Candidate login and responses to questions

[0185] Output: Candidate response data

[0186] Step 6: Parse the response and generate the next question

[0187] The server analyzes the candidate's responses and generates the next question based on the content in real time.

[0188] For example, if you answer "I have experience with Python and Java," questions like "What specific projects have you used those languages ​​in?" will be generated.

[0189] Input: Candidate response data

[0190] Output: Next question

[0191] Step 7: Gather feedback

[0192] After the interview, the user uses the terminal to enter feedback.

[0193] Feedback may include an evaluation of the interview as a whole and an evaluation of specific questions and responses.

[0194] The terminal sends the feedback to the server.

[0195] Input: User feedback information

[0196] Output: Feedback data

[0197] Step 8: Feedback analysis and optimization

[0198] The server analyzes the collected feedback and interview performance data.

[0199] The server optimizes the virtual interviewer's questioning style and evaluation criteria based on the analysis results.

[0200] For example, if you receive feedback that there are not enough questions to assess logical thinking, add a question such as "Please explain the specific steps you took to solve the problem" to your next interview.

[0201] Input: Feedback data and interview performance data

[0202] Output: Optimized question style and evaluation criteria

[0203] Step 9: Store and analyze the interview data

[0204] The server collects and stores data from all interview sessions.

[0205] The server analyses the collected data and provides the user with insight into the interview process.

[0206] Input: Interview session data

[0207] Output: Interview process insights and reports

[0208] Step 10: Dynamically Adjust System Resources

[0209] The server dynamically adjusts system resources according to the company's adoption volume.

[0210] When multiple interviews are conducted at once, the server automatically increases computing resources and distributes the load to ensure stable terminal operation.

[0211] Input: Change in recruitment volume

[0212] Output: Dynamically adjusted system resources

[0213] (Application example 1)

[0214] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0215] In modern factories, accurately evaluating the capabilities of automated robots and efficiently training them are key challenges. Robots with diverse functions in particular need to have their skills evaluated and improved appropriately, but existing systems make it difficult to customize individual evaluation criteria and training scenarios, making it difficult to ensure an optimal training process.

[0216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0217] In this invention, the server includes a means for generating a customized virtual trainer based on a user's request, a means for conducting real-time interaction with the trainee device and performing skill evaluation, and a means for collecting and analyzing training data to provide an optimized training experience, thereby enabling accurate evaluation of the robot's capabilities and providing adaptive and efficient training.

[0218] A "user" is an individual or group that operates the system or requests customization.

[0219] A "virtual trainer" is a virtual role model that is generated based on a user's request and performs skill evaluation and training on the trainee device.

[0220] "Training equipment" refers to machinery and equipment that is the subject of training and evaluation, such as robots and automated operating devices used in factories.

[0221] "Real-time interaction" refers to the process in which the virtual trainer interacts directly with the trainee device in real time, exchanging questions and instructions in real time.

[0222] "Skills evaluation" is the act of measuring and evaluating abilities such as the skill to operate the training equipment, reaction speed, and error handling ability.

[0223] "Training Data" means data collected during a training session, including the response, performance, and feedback of the trained device.

[0224] An "optimized training experience" is a training session that is customized to the characteristics of the user and the equipment being trained, and is delivered efficiently and effectively.

[0225] "Feedback" refers to ratings and comments provided by users and those involved in the training, which are used to improve the virtual trainer and optimize the training.

[0226] The "question style" refers to the format or method of questions that the virtual trainer asks the trainee device.

[0227] "Evaluation criteria" are the standards or metrics used to evaluate the trainee's skills.

[0228] The present invention provides a system for effectively training and evaluating robots in factories. An embodiment of the system will be described in detail below.

[0229] The system primarily consists of a user, a terminal, and a server. The user, such as a factory staff member, inputs the characteristics and evaluation criteria of the robot they want to train. The terminal is a computer, tablet, or smartphone operated by the user, and is responsible for collecting information from these and sending it to the server. The server uses a generative AI model to generate a virtual trainer and evaluates the robot's operation and performs training in real time.

[0230] Program processing

[0231] The server performs the process as follows:

[0232] 1. Receive user customization requests

[0233] The user inputs the skills they want to evaluate (e.g., operation skills, reaction speed, error handling ability) through the terminal, and this information is sent to the server.

[0234] 2. Creating a Virtual Trainer

[0235] The server uses the OpenAI API to generate prompts based on the user's requests. For example, the customization request is converted into a prompt, such as "User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, error handling ability." The server then uses the generative AI model to generate a virtual trainer.

[0236] 3. Training commencement and evaluation

[0237] The server activates a virtual trainer, which interacts with the trainee in real time. The virtual trainer presents the robot with a series of questions and tasks, and collects and evaluates its responses. For example, it runs scenarios to evaluate skills such as object recognition, obstacle avoidance, and efficient route planning.

[0238] 4. Data Collection and Optimization

[0239] Data collected during the training session (robot responses, performance data, user feedback, etc.) is stored and analyzed on a server. Based on the results of this analysis, the virtual trainer's questioning style and evaluation criteria are optimized.

[0240] Hardware and software used

[0241] Hardware: Factory robots (e.g., transport robots, assembly robots), user-operated devices (computers, tablets, smartphones)

[0242] Software: OpenAI API, Python program

[0243] Specific examples

[0244] When training a transport robot in a factory, the user inputs evaluation skills using a terminal. For example, if they want to evaluate "operation skills, reaction speed, and error handling ability," this is converted into prompt statements and sent to the server. The virtual trainer presents the robot with a scenario such as, "To evaluate your operation skills, please recognize and transport objects accurately." The robot's behavior is evaluated in real time, and after training is complete, it is optimized based on the feedback.

[0245] Example prompt sentence:

[0246] User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, error handling ability

[0247] Scenario: The virtual trainer sets up an evaluation scenario for the transport robot and records its responses in real time.

[0248] This embodiment allows accurate evaluation of the capabilities of robots in a factory and effective training.

[0249] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0250] Step 1:

[0251] The user inputs the skills they want to evaluate and the characteristics of the robot they are training through their device. For example, the user specifies "modelXYZ" as the robot they want to evaluate for its "operation skills, reaction speed, and error handling ability." This input information is sent from the device to the server.

[0252] Step 2:

[0253] The server generates a prompt based on the received user customization request. The generated prompt is "User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, and error handling ability." This provides the specific data required to generate a virtual trainer.

[0254] Step 3:

[0255] The server generates a virtual trainer using the generative AI model. Specifically, it calls the OpenAI API and provides prompts as input data. The API response generates scripts and scenarios for the virtual trainer, which are then stored on the server.

[0256] Step 4:

[0257] The server activates the virtual trainer to start training. The server sends a training start command to the terminal, and the terminal sends a training start signal to the trained device (robot). At this point, the virtual trainer begins to develop a scenario suitable for the model XYZ robot.

[0258] Step 5:

[0259] A virtual trainer sets up evaluation scenarios for the robot in real time, presenting questions and tasks to the robot, which then perform specific tasks such as object recognition, obstacle avoidance, and efficient route planning. This process generates data on the robot's responses.

[0260] Step 6:

[0261] The server collects the robot's response data generated in real time, including task completion time, number of errors, and performance indicators. This data is stored on the server as training results.

[0262] Step 7:

[0263] The user inputs feedback on the training results through the device. For example, they can provide specific feedback such as, "The robot's reaction speed is appropriate, but its obstacle avoidance is insufficient." This feedback is then sent from the device to the server.

[0264] Step 8:

[0265] The server analyzes user feedback and collected training data to optimize the virtual trainer's questioning style and evaluation criteria, thereby providing a more adaptive and efficient scenario for the next training session.

[0266] Step 9:

[0267] The server updates the optimized virtual trainer profile and makes it available for the next use, thereby achieving continuous training and assessment improvement.

[0268] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0269] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion engine. This system has various functions, such as generating a customized virtual interviewer based on the user's request, conducting real-time Q&A, analyzing and providing interview data, and optimizing question styles based on feedback. Furthermore, by incorporating an emotion engine, the system evaluates the candidate's emotional state, providing more human-like interaction and comprehensive evaluation.

[0270] 1. Interviewer generation and customization

[0271] The user inputs a customization request to generate an interviewer for a specific industry and job type through the device's operation screen. For example, if the goal is to hire a software engineer, the user inputs the skills (programming skills and algorithm knowledge) and characteristics (problem-solving ability and communication ability) they want to evaluate. The device collects this information and sends it to the server as a customization request.

[0272] The server uses a generative AI model to generate a virtual interviewer profile based on the user's input data. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0273] 2. Real-time Q&A

[0274] The user sends an interview invitation to the candidate and schedules the interview. The terminal is prepared so that the candidate can log in at the specified date and time. The server activates the virtual interviewer at the start time of the interview, and also activates the emotion engine.

[0275] The server asks the first question via a virtual interviewer, and the candidate inputs their response via their device. The device then sends this response to the server, where the emotion engine analyzes the candidate's facial expressions, tone of voice, and choice of words to evaluate their emotional state. For example, if a candidate answers, "I have experience with Python and Java," the tone of voice and facial expressions will be used to evaluate whether they are speaking with confidence.

[0276] The server takes into account the analysis results of the emotion engine and generates the next question in real time. The next question is adjusted according to the candidate's emotional state. For example, if the emotion engine determines that the candidate is nervous, the server will present a relaxing question such as "Tell us about your specific project experience."

[0277] This question and answer process is repeated until all questions are answered, and all data collected by the server is recorded.

[0278] 3. Evolving learning functions

[0279] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the server will reflect this in the AI ​​model and ask more detailed questions in the next interview.

[0280] 4. Interview data analysis

[0281] All data from the interview session is collected and stored by the server, including the candidate's responses, evaluation scores, collected sentiment data, and feedback. The server analyzes this data and provides insights to the user, such as average technical skill scores across multiple candidates or trends in specific skill deficiencies, and provides these to the user in the form of a report.

[0282] 5. Providing flexible and scalable solutions

[0283] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the computing resources will be automatically increased. This allows for load balancing to ensure stable operation of terminals.

[0284] Users can customize the settings from their devices for different job types and departments. For example, they can create virtual interviewers for marketing and sales positions, as well as software engineers. The server generates virtual interviewers based on each requirement and helps ensure the interview process goes smoothly.

[0285] The above is an embodiment of the present invention, which enables companies to efficiently and effectively recruit talented personnel by using a virtual interviewer incorporating an emotion engine.

[0286] The processing flow will be explained below.

[0287] 1. Interviewer generation and customization

[0288] Step 1:

[0289] The user inputs a customization request for generating an interviewer corresponding to a specific industry and job type via the operation screen of the terminal.

[0290] Step 2:

[0291] The terminal collects the information input by the user and transmits it to the server as a customization request.

[0292] Step 3:

[0293] The server analyzes the requests received from users and uses a generative AI model to generate a profile for the virtual interviewer, which includes question patterns and evaluation criteria appropriate for the industry and job type.

[0294] Step 4:

[0295] The server stores the generated profile of the virtual interviewer, and the interview preparation is complete.

[0296] 2. Real-time Q&A

[0297] Step 1:

[0298] The user sends interview invitations to candidates and schedules interview dates.

[0299] Step 2:

[0300] The terminal will be ready for the candidate to log into the system at the specified date and time.

[0301] Step 3:

[0302] The server activates the virtual interviewer and the emotion engine in accordance with the start time of the interview.

[0303] Step 4:

[0304] The server asks the first question via a virtual interviewer.

[0305] Step 5:

[0306] The candidate enters the response through a terminal.

[0307] Step 6:

[0308] The terminal transmits the candidate's response to the server.

[0309] Step 7:

[0310] The server analyzes the candidate's responses in real time and generates the next question based on the content, while an emotion engine is initiated to analyze the candidate's facial expressions, tone of voice, choice of words, etc. to evaluate their emotional state.

[0311] Step 8:

[0312] The server considers the results of the emotion engine's evaluation and adjusts the content of questions and pace of the interview depending on the candidate's emotional state. For example, if it determines that the candidate is nervous, it selects questions that will help them relax, such as "Tell us about your specific project experience."

[0313] Step 9:

[0314] The terminal presents the next question received from the server to the candidate.

[0315] Step 10:

[0316] This process is repeated until all questions have been completed.

[0317] 3. Evolving learning functions

[0318] Step 1:

[0319] After the interview, the user enters feedback through the system.

[0320] Step 2:

[0321] The terminal collects feedback from the user and transmits it to the server.

[0322] Step 3:

[0323] The server analyzes the collected feedback and interview performance data. Based on the analysis results, the AI ​​model automatically adjusts the questioning style and evaluation criteria. For example, if the feedback states that "there were not enough questions to evaluate logical thinking," the AI ​​model will add more detailed questions to the next interview.

[0324] 4. Interview data analysis

[0325] Step 1:

[0326] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, emotion data from the emotion engine, and feedback.

[0327] Step 2:

[0328] The server analyzes the collected data and provides the user with insights into the interview process, such as analyzing the average technical skill scores of multiple candidates or trends in deficiencies in specific skills, and presents the results to the user in a report.

[0329] Step 3:

[0330] The terminal displays the analysis results provided by the server to the user.

[0331] 5. Providing flexible and scalable solutions

[0332] Step 1:

[0333] The server dynamically adjusts system resources according to the company's hiring volume. For example, if a large number of interviews are being conducted at once, the server automatically increases computing resources.

[0334] Step 2:

[0335] The terminals perform load balancing to ensure stable operation of the system.

[0336] Step 3:

[0337] Users can customize settings for different job types and departments from their terminals.

[0338] Step 4:

[0339] The server generates an appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[0340] This allows companies to use virtual interviewers with an embedded emotion engine to efficiently and effectively recruit top talent.

[0341] Example 2

[0342] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0343] Conventional interview systems have difficulty efficiently evaluating a large number of candidates at once, and customization for specific industries or job types is difficult. Furthermore, they lack the ability to adjust questions in real time to take into account the candidate's emotional state, resulting in inconsistent interview quality. Furthermore, they lack the learning functionality to appropriately reflect feedback and evolve questioning styles and evaluation criteria.

[0344] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for generating a customized virtual interviewer based on user requests; means for conducting real-time dialogue with candidates and conducting Q&A sessions; means for collecting and analyzing interview data to provide an optimized interview experience; means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results; means for evaluating the candidate's emotional state using an emotion analysis engine and adjusting the content of questions in real time; and means for the server to dynamically adjust resources and perform load balancing when multiple interviews are conducted simultaneously. This simplifies customization for specific industries and job types and enables real-time question adjustment taking into account the candidate's emotional state. Furthermore, the quality of interviews can be improved through efficient analysis of interview data and reflection of feedback.

[0345] A "user" is an entity that interacts with the system to customize a virtual interviewer and coordinate and oversee interviews with candidates.

[0346] "Terminal" means the device used by a user or candidate to access the system, including a PC, smartphone, tablet, etc.

[0347] The "server" is a central processing unit that manages and operates the entire system, generates virtual interviewers, answers questions in real time, analyzes data, and evaluates emotional states.

[0348] A "virtual interviewer" is a virtual presence that is customized based on user requirements using generative AI models to interact with candidates and ask questions in real time.

[0349] An "emotion analysis engine" is a software component that analyzes a candidate's facial expressions, tone of voice, choice of words, etc. to assess their emotional state.

[0350] A "generative AI model" is an artificial intelligence algorithm that generates a profile of a virtual interviewer based on user input data and creates question patterns and evaluation criteria.

[0351] A "customization request" is a requirement or condition provided by a user to generate a virtual interviewer suited to a particular industry or job type.

[0352] "Real-time interaction" refers to the process in which a virtual interviewer asks and responds to candidates in real time.

[0353] "Load balancing" is the process by which a server dynamically adjusts computing resources and distributes them to maintain system stability when multiple interviews are running simultaneously.

[0354] "Feedback" refers to opinions and evaluations provided by users through the system after the interview, and is used to optimize questioning style and evaluation criteria.

[0355] "Questioning style" refers to the format and method of asking questions used by the virtual interviewer.

[0356] "Rating criteria" are the standards or metrics that the virtual interviewer uses to evaluate the candidate's skills and attributes.

[0357] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion analysis engine. This system has various functions, such as generating a customized virtual interviewer based on the user's request, conducting real-time Q&A, analyzing and providing interview data, and optimizing question styles based on feedback. Furthermore, by incorporating an emotion analysis engine, it evaluates the candidate's emotional state, providing a more human-like interaction and comprehensive evaluation.

[0358] First, the user inputs a customization request to generate an interviewer for a specific industry and job type through the device's operation screen. This operation screen is provided on devices such as PCs, tablets, and smartphones. For example, the user might input a request such as, "If I'm looking to hire a software engineer, I'd like to evaluate Python and Java programming skills as the skills I want to assess, and problem-solving ability and communication ability as the attributes." The device receives this request and sends it to the server.

[0359] The server uses a generative AI model to generate a virtual interviewer profile based on the user's customization request. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0360] As a specific prompt, the server sends the following request to the generative AI model: "Please generate a virtual interviewer to conduct job interviews for software engineers. The skills I want to evaluate are programming skills in Python and Java. I also want to evaluate problem-solving ability and communication ability as traits."

[0361] After preparing for the interview, the user uses the virtual interviewer's profile to send an interview invitation to the candidate. The device prepares the necessary equipment so that the candidate can log in at the specified date and time. When the interview starts, the server activates the virtual interviewer and the sentiment analysis engine as well. This allows for real-time question and answer sessions.

[0362] The server asks the candidate an initial question via a virtual interviewer, and the candidate enters their response via their device. The device then sends the response data to the server, where an emotion analysis engine analyzes the candidate's facial expressions, tone of voice, and word choice in real time to assess their emotional state. For example, if a candidate answers, "I have experience with Python and Java," the server adjusts the next question based on the results of the emotion analysis engine. If the candidate is nervous, it can ask questions that will help them relax.

[0363] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the AI ​​model will be informed to add more detailed questions in the next interview.

[0364] Additionally, all data collected during the interview session is stored and analyzed by the server, including the candidate's responses, evaluation scores, collected sentiment data, feedback, etc. The server uses this data to provide insights to the user and provides reports analyzing trends in specific skill deficiencies and average technical skill scores across multiple candidates.

[0365] The server also dynamically adjusts resources and distributes the load to ensure stable system operation even when multiple interviews are conducted at once. This allows terminals to operate stably and multiple interviews to proceed smoothly. This allows companies to efficiently and effectively recruit top talent using virtual interviewers equipped with an emotion analysis engine.

[0366] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0367] Step 1:

[0368] The user inputs a customization request to generate a virtual interviewer for a specific industry and job type through the device's operation screen. For example, for the purpose of hiring a software engineer, a specific request could be input, such as "The skills I want to evaluate are Python and Java programming skills, and the characteristics I want to evaluate are problem-solving ability and communication ability." The input data is sent from the device to the server as a customization request. Input: Industry, Job Type, Evaluation Skills, Evaluation Characteristics | Output: Customization Request

[0369] Step 2:

[0370] The terminal formats the customization request entered by the user and sends it to the server via the network. A specific prompt would be, "Generate a virtual interviewer to conduct a software engineer job interview. The skills we want to evaluate are programming skills in Python and Java. We also want to evaluate problem-solving ability and communication ability as traits." Input: Customization request | Output: Prompt

[0371] Step 3:

[0372] The server receives the customization request and uses the generative AI model to generate a virtual interviewer profile. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is stored in a database. Input: Prompt sentence | Output: Virtual interviewer profile

[0373] Step 4:

[0374] The user uses the generated virtual interviewer profile to send interview invitations to candidates and arrange interview dates. The terminal prepares the candidate for login and prepares the necessary interface. Input: Virtual interviewer profile | Output: Interview invitation

[0375] Step 5:

[0376] The server activates the virtual interviewer at the start time of the interview. The emotion engine is also activated and ready. The server generates the initial questions and asks them to the candidate. Input: Interview invitation | Output: Initial questions

[0377] Step 6:

[0378] Candidates input their answers through their devices. The devices then send the answers to the server, which then uses an emotion engine to analyze the answers. For example, if a candidate answers "I have experience with Python and Java," the server evaluates whether they are speaking with confidence based on their tone of voice and facial expressions. Input: Candidate's answers | Output: Analysis data

[0379] Step 7:

[0380] The server generates the next question based on the results of the sentiment analysis engine. Depending on the candidate's level of nervousness, the server adjusts the questions to present a more relaxed response. The question and answer process is repeated until all questions have been completed. Input: Analysis data | Output: Next question

[0381] Step 8:

[0382] After the interview, the user writes down their feedback, which is then sent to the server. The server analyzes the feedback and optimizes the question style and evaluation criteria for the next interview. Input: Feedback | Output: Optimized question style

[0383] Step 9:

[0384] The server collects and stores data from the entire interview session, performs analysis, and provides insights to the user, including average technical skill scores and trends in specific skill deficiencies. Input: Interview data, analysis data | Output: Report

[0385] Step 10:

[0386] The server dynamically adjusts resources according to the company's recruitment volume. When multiple interviews are conducted simultaneously, the server increases computing resources and distributes the load. This ensures stable system operation. Input: Interview schedule | Output: Dynamic resource adjustment

[0387] (Application example 2)

[0388] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0389] Conventional virtual interviewer systems have difficulty taking into account emotional changes during interviews with candidates, resulting in a uniform interview experience. Furthermore, the interviewer's Q&A and evaluations are fixed, resulting in insufficient optimization of the interview. Furthermore, there is insufficient automation of interview data analysis and feedback, which can lead to delays in the evolution of interviewer training and evaluation criteria.

[0390] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0391] In this invention, the server includes means for generating a customized virtual interviewer based on user requests, means for conducting real-time dialogue with candidates and conducting Q&A sessions, means for collecting and analyzing interview data to provide an optimized interview experience, means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results, means for analyzing emotional evaluations, and means for adjusting questions based on the analysis results. This enables interactive interviews that take into account the candidate's emotional state, resulting in a high-quality interview experience and efficient talent evaluation.

[0392] "User" refers to an individual or organization that uses this system.

[0393] A "virtual interviewer" refers to a program that uses generative AI models and emotion engines to mimic the role of a real-life interviewer.

[0394] "Real-time interaction" refers to simultaneous interaction between the interviewer and candidate.

[0395] "Question and answer session" refers to the process in which the interviewer asks questions and the candidate answers them.

[0396] "Interview data" refers to information obtained during an interview, such as the candidate's responses, facial expressions, and tone of voice.

[0397] "Analysis" refers to the process of evaluating the collected interview data and deriving meaningful results.

[0398] An "optimized interview experience" refers to an interview process that is tailored to the candidate's emotional state and reactions.

[0399] "Feedback" refers to the evaluation and opinions provided by the user after the interview is completed.

[0400] "Questioning style" refers to the format or pattern of questions used by the interviewer.

[0401] "Evaluation criteria" refers to the standards or criteria for evaluating a candidate's responses or performance.

[0402] "Emotional assessment" refers to the process of analyzing a candidate's facial expressions and tone of voice to determine their emotional state.

[0403] "Analysis results" refer to the specific conclusions or findings obtained after conducting an emotion assessment.

[0404] "Question tailoring" refers to the process of changing or selecting the next question based on the candidate's emotional state.

[0405] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion engine. This system generates a customized virtual interviewer based on the user's request and has functions such as real-time question and answer sessions, analysis and provision of interview data, and optimization of question styles based on feedback. In addition, by incorporating an emotion engine, it evaluates the candidate's emotional state and provides human-like interaction and comprehensive evaluation.

[0406] System Configuration

[0407] 1. User:

[0408] The user inputs a customization request for generating an interviewer corresponding to a specific industry and job type through the operation screen of the terminal.

[0409] 2. Terminal:

[0410] The terminal collects customization requests from users and sends them to a server. Wearable devices such as smart glasses are used for training and interviewing new operators.

[0411] 3. Server:

[0412] The server uses a generative AI model based on the user's input data to generate a virtual interviewer profile, which includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0413] Real-time Q&A

[0414] The server activates the virtual interviewer at the start of the interview, and also activates the emotion engine. The server asks the first question through the virtual interviewer, and the candidate inputs their response through their terminal. The terminal sends this response to the server, and the emotion engine analyzes the candidate's facial expressions, tone of voice, choice of words, etc. to evaluate their emotional state.

[0415] For example, if a candidate answers, "I have experience operating machines," the emotion engine evaluates whether they are speaking with confidence based on their tone of voice and facial expressions. The server takes this analysis into account and generates the next question in real time. For example, if the emotion engine determines that the candidate is nervous, the server will present a question that will help them relax, such as, "Tell us about your specific project experience."

[0416] Learning Function

[0417] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the server will reflect this in the AI ​​model and ask more detailed questions in the next interview.

[0418] Data analysis and provision

[0419] All data from the interview session is collected and stored by the server, including the candidate's responses, evaluation scores, collected sentiment data, and feedback. The server analyzes this data and provides insights to the user. For example, it analyzes the average technical skill scores of multiple candidates and trends in specific skill deficiencies, and provides this to the user as a report.

[0420] Providing flexible and scalable solutions

[0421] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the server can automatically increase computing resources and distribute the load to ensure stable operation of the terminals.

[0422] Examples and prompts

[0423] For example, if a new factory operator answers, "I would like to learn how to operate a new machine," during training for new employees, the virtual interviewer will analyze their tone of voice and facial expression, and if it determines that they are nervous, it will put them at ease by asking them the next question, "Please explain the specific operating procedures."

[0424] Example prompt sentence:

[0425] You are a virtual training assistant helping train new employees in a factory. Based on the responses provided by the candidate, you use an emotion engine to analyze their emotional state and tailor the next questions you ask.

[0426] This enables interactive interviews that take into account the candidate's emotional state, resulting in a high-quality interview experience and efficient talent evaluation.

[0427] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0428] Step 1:

[0429] The user operates the device to input customization requests to generate a virtual interviewer for a specific industry and job type. The data input by the user includes the skills (e.g., programming skills, knowledge of machine operations) and characteristics (e.g., problem-solving ability, communication ability) to be evaluated. The input data is sent to the server in JSON format.

[0430] Step 2:

[0431] The device collects customization requests from the user and sends them to the server. The data sent as the customization request includes the user's input information. Upon receiving this data, the server uses a generative AI model to generate a profile for the virtual interviewer. The profile includes question patterns and evaluation criteria appropriate for the industry and job type.

[0432] Step 3:

[0433] The server saves the generated virtual interviewer profile and prepares for the interview. The profile saves specific questions and evaluation criteria in template format. The saved profile is used in subsequent processing.

[0434] Step 4:

[0435] After the interview date is arranged, the terminal sends an interview invitation to the candidate, and prepares the candidate so that they can log in at the specified date and time. The server verifies the login information and activates the virtual interviewer at the start time of the interview.

[0436] Step 5:

[0437] When the interview begins, the server asks the first question via the virtual interviewer, and the candidate enters their response via the terminal. The entered response data is sent in text format to the server, which then receives the candidate's response and analyzes the data using a natural language processing engine.

[0438] Step 6:

[0439] The server receives the candidate's responses and uses an emotion engine to analyze the candidate's facial expressions, tone of voice, and word choice. Based on the analysis results, the server evaluates the candidate's emotional state (e.g., nervous, confident, confused). The evaluation data is used to generate the next questions in real time.

[0440] Step 7:

[0441] Taking into account the analysis results of the emotion engine, the server generates the next question in real time. For example, if the server determines that the candidate is nervous, it will present a question that will help them relax, such as "Tell us about your specific project experience." The generated question is displayed to the candidate via their device.

[0442] Step 8:

[0443] The question and answer process is repeated until all questions are completed. The server records all collected data and stores all data from the interview session, including the candidate's responses, evaluation scores, collected emotional data, and feedback.

[0444] Step 9:

[0445] After the interview, the user enters feedback through the system. The terminal collects the feedback and sends it to the server. The server analyzes the collected feedback and optimizes the question style and evaluation criteria for the next interview, thereby improving the accuracy and effectiveness of the virtual interviewer.

[0446] Step 10:

[0447] The server generates an interview report based on the analysis results and provides it to the user. The report includes the average technical skill scores of multiple candidates, trends in specific skill deficiencies, and an overall evaluation of the interview. Users can use this report to optimize their recruitment activities.

[0448] The above are the specific processing steps for implementing the present invention, which enable interactive interviews that take into account the candidate's emotional state, thereby achieving a high-quality interview experience and efficient talent evaluation.

[0449] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0450] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0451] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0452] [Second embodiment]

[0453] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0454] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0455] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0456] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0457] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0458] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0459] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0460] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0461] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0462] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0463] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0464] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0465] The present invention will be described below as a specific method for implementing the invention, with reference to the operation of a system using a user, a terminal, and a server. The invention is implemented based on the following configuration.

[0466] 1. Interviewer generation and customization

[0467] 1-1: Receive customization requests

[0468] Users use the system to submit customization requests to generate interviewers for specific industries and job types. For example, if the user is looking to hire a software engineer, the user inputs the skills (e.g., programming skills and algorithm knowledge) and characteristics (e.g., problem-solving ability and communication ability) they want to evaluate on the device's operation screen. The device then collects this information and sends it to the server as a customization request.

[0469] 1-2: Generate interviewers

[0470] The server analyzes the customization request sent by the user and generates a virtual interviewer using a generative AI model based on the analysis. The virtual interviewer has question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0471] 2. Real-time Q&A

[0472] 2-1: Start of the interview

[0473] The user sends an interview invitation to the candidate and schedules the interview. The terminal is prepared so that the candidate can log in to the system at the specified date and time. The server activates the virtual interviewer at the start time of the interview.

[0474] 2-2: Providing questions and recording responses

[0475] The server asks the candidate an initial question via a virtual interviewer. For example, "Tell me about your programming experience." When the candidate enters their response via their terminal, it is sent to the server. The server analyzes the candidate's response and generates the next question in real time based on the content. For example, if the candidate answers "I have experience with Python and Java," the server generates the next question in real time: "In what specific projects have you used those languages?" This allows for a detailed evaluation of the candidate's skills and aptitude.

[0476] 3. Evolving learning functions

[0477] 3-1: Collecting and sending feedback

[0478] After the interview, the user enters feedback through the system, including an evaluation of the overall interview and evaluation of specific questions and responses. The terminal collects this feedback and sends it to the server.

[0479] 3-2: Optimizing question style and evaluation criteria

[0480] The server analyzes the collected feedback and interview performance data. For example, if feedback indicates that there are not enough questions to evaluate the candidate's logical thinking, the system can use this information to adjust the questioning style and add a question such as "Please explain the specific steps you took to solve the problem" to the next interview. In this way, the virtual interviewer can continuously evolve and provide more effective interviews.

[0481] 4. Interview data analysis

[0482] 4-1: Collection of interview data

[0483] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback.

[0484] 4-2: Analyzing data and providing insights

[0485] The server analyzes the collected data and provides the user with insights into the interview process, such as analyzing the average technical skill scores of multiple candidates or specific skill deficiencies, and generates a report that is provided to the user, allowing the user to gain specific insights to further optimize the hiring process.

[0486] 5. Providing flexible and scalable solutions

[0487] 5-1: Scaling up

[0488] The server dynamically adjusts system resources according to the company's hiring volume. For example, if a large number of interviews are being conducted at once, the necessary computing resources will be automatically increased. The server also distributes the load to ensure stable operation of terminals.

[0489] 5-2: Meeting different hiring needs

[0490] Users can use their devices to customize settings for different job types and departments. For example, they can set up the system to generate virtual interviewers that are optimal for each job type, so that it can accommodate not only software engineers but also marketing and sales positions. The server generates the appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[0491] The above is an embodiment of the present invention, which enables companies to recruit talented personnel efficiently and effectively.

[0492] The processing flow will be explained below.

[0493] 1. Interviewer generation and customization

[0494] Step 1:

[0495] The user inputs the industry, job type, and skills and characteristics they want to evaluate through the device's operation screen.

[0496] Step 2:

[0497] The terminal collects input information from the user and transmits it to the server as a customization request.

[0498] Step 3:

[0499] The server analyzes the received request and uses a generative AI model to generate a profile for the virtual interviewer, which includes question patterns and evaluation criteria appropriate for the industry and job type.

[0500] Step 4:

[0501] The server stores the generated virtual interviewer profile and the interview is ready.

[0502] 2. Real-time Q&A

[0503] Step 1:

[0504] The user sends interview invitations to candidates and schedules interviews.

[0505] Step 2:

[0506] The terminal will be ready for the candidate to log into the system at the appointed date and time.

[0507] Step 3:

[0508] The server activates the virtual interviewer at the start time of the interview.

[0509] Step 4:

[0510] The server asks the candidate an initial question via a virtual interviewer.

[0511] Step 5:

[0512] The candidate enters their response through a terminal.

[0513] Step 6:

[0514] The terminal sends the candidate's response to the server.

[0515] Step 7:

[0516] The server analyzes the candidate's responses and generates subsequent questions in real time based on their content. For example, if a question about technical skills follows, the server generates a question such as, "What specific projects have you used those languages ​​in?"

[0517] Step 8:

[0518] The server generates the next question and sends it to the terminal, which then presents it to the candidate.

[0519] Step 9:

[0520] This process is repeated until all questions have been completed.

[0521] 3. Evolving learning functions

[0522] Step 1:

[0523] After the interview, the user enters feedback through the system.

[0524] Step 2:

[0525] The device collects the feedback and sends it to the server.

[0526] Step 3:

[0527] The server analyzes the collected feedback and interview performance data. Based on the analysis results, the question style and evaluation criteria are automatically adjusted according to the AI ​​model. For example, if feedback indicates that there are a lack of questions that evaluate logical thinking, relevant question patterns are added.

[0528] 4. Interview data analysis

[0529] Step 1:

[0530] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback.

[0531] Step 2:

[0532] The server analyzes the collected data and provides the user with insight into the interview process, such as average technical skill scores or identifying skill gaps, and generates a report of the results.

[0533] Step 3:

[0534] The terminal displays the analysis results provided by the server to the user.

[0535] 5. Providing flexible and scalable solutions

[0536] Step 1:

[0537] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the computing resources will be automatically increased.

[0538] Step 2:

[0539] Terminals distribute the load to ensure stable operation of the system.

[0540] Step 3:

[0541] Users can customize settings for different job types or departments from their terminals.

[0542] Step 4:

[0543] The server generates an appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[0544] The above are the specific processing steps of the present invention, and this flow enables companies to recruit talented personnel efficiently and effectively.

[0545] Example 1

[0546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0547] In the modern recruitment process, there are many challenges in candidate evaluation and interview efficiency. In particular, evaluations are often influenced by the interviewer's subjectivity, preparing for and conducting interviews requires a great deal of time and resources, and continuous improvement based on feedback is difficult. There is a need to solve these challenges and provide a more effective and efficient interview process.

[0548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0549] In this invention, the server includes a means for generating a customized virtual interviewer based on user requests, a means for conducting real-time dialogue with candidates and conducting Q&A sessions, and a means for collecting and analyzing interview data to provide an optimized interview experience. This streamlines the interview process and enables fair and objective candidate evaluation. Furthermore, the virtual interviewer's questioning style and evaluation criteria can be automatically learned using user feedback and interview results, enabling continuous improvement. Furthermore, because the customized virtual interviewer is generated using a generative AI model, it is easy to create a virtual interviewer with question patterns and evaluation criteria suited to specific industries and job types.

[0550] A "user request" is a request by a user to customize the virtual interviewer based on a particular industry, job type, and assessment skills when generating the virtual interviewer.

[0551] A "customized virtual interviewer" is a virtual interviewer with question patterns and evaluation criteria suited to the needs of a specific industry or job type, according to the user's request.

[0552] A "generative AI model" is an artificial intelligence model that analyzes user requirements and generates a customized virtual interviewer based on those requirements.

[0553] "Real-time dialogue" refers to the process of an interview in which the candidate and virtual interviewer interactively exchange questions and answers, with immediate responses and feedback.

[0554] A "question and answer" is a series of interactions in which a virtual interviewer asks a candidate questions and the candidate responds.

[0555] "Interview data" refers to data collected during an interview, including information such as the candidate's responses, evaluation scores, and feedback.

[0556] An "optimized interview experience" refers to an interview process that improves the quality and efficiency of interviews by analyzing interview data and utilizing feedback.

[0557] "Feedback" refers to the evaluation of the entire interview provided by the user after the interview is completed, as well as evaluation comments on specific questions and answers.

[0558] "Questioning style" is a concept that refers to the format and manner in which a virtual interviewer asks questions to a candidate.

[0559] "Evaluation criteria" are indicators or standards that the virtual interviewer uses to evaluate the candidate's responses, and are set based on the user's requests.

[0560] A "profile" refers to a generated set of information about a virtual interviewer, including data related to interview preparation.

[0561] "Analysis" is the process of analyzing the collected interview data and generating insights and recommendations based on the results.

[0562] The present invention is an interview support system implemented using a user, a terminal, and a server, and a specific implementation method thereof will be described below. Implementing the invention requires cooperation between hardware and software.

[0563] 1. Interviewer generation and customization

[0564] To generate a virtual interviewer for a specific industry and job type, the user inputs customization requests using the device's operation screen. Examples of input include evaluation items such as "programming skills," "algorithm knowledge," "problem-solving ability," and "communication ability." The device collects this input information and sends it to the server as a customization request.

[0565] The server analyzes the received customization request and uses a generative AI model to generate a customized virtual interviewer with question patterns and evaluation criteria suited to the specific industry and job type. The generated virtual interviewer profile is stored on the server and is ready for the interview.

[0566] 2. Real-time Q&A

[0567] The user sends an interview invitation to the candidate from the terminal and arranges the interview date. The terminal makes the necessary preparations so that the candidate can log in to the system at the specified date and time. The server activates the virtual interviewer at the start time of the interview.

[0568] When the interview begins, the server provides the candidate with the first question through the virtual interviewer. A specific example question is, "Tell me about your programming experience." The candidate enters their response through their terminal, which is then sent to the server. The server analyzes the candidate's response and generates the next question in real time based on the content. For example, if the candidate answers, "I have experience with Python and Java," the server generates a question such as, "In what specific projects have you used those languages?"

[0569] 3. Evolving learning functions

[0570] After the interview, the user enters feedback through the system. The feedback includes an evaluation of the entire interview and evaluation of specific questions and answers. The terminal collects this feedback and sends it to the server.

[0571] The server analyzes the collected feedback and interview performance data to optimize the virtual interviewer's questioning style and evaluation criteria. For example, if feedback indicates that there are not enough questions to evaluate the candidate's logical thinking, the server will add a question such as "Please explain the specific steps you took to solve the problem" to the next interview. In this way, the virtual interviewer continuously evolves and provides more effective interviews.

[0572] 4. Interview data analysis

[0573] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback. The server analyzes the collected data and provides users with insights into the interview process. For example, the average technical skill scores of multiple candidates and specific skill deficiencies are analyzed and provided to users as a report. This allows users to gain specific insights to further optimize the hiring process.

[0574] 5. Providing flexible and scalable solutions

[0575] The server dynamically adjusts system resources according to the company's hiring volume. For example, when conducting a large number of interviews at once, it automatically increases the necessary computing resources and distributes the load to ensure stable terminal operation. Users can also use their terminals to customize settings for different job types and departments. For example, it is possible to set up the system to generate virtual interviewers optimized for each job type, so that it can accommodate not only software engineers but also marketing and sales positions. The server generates appropriate virtual interviewers based on each request and supports the smooth conduct of interviews.

[0576] Specific examples

[0577] An example prompt might be, "What skills are necessary to generate software engineer interviewers?", to which the system generates specific answers such as "programming skills, knowledge of algorithms, problem-solving ability, and communication skills."

[0578] By following the above procedure, the present invention can be implemented, enabling companies to recruit talented personnel efficiently and effectively.

[0579] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0580] Step 1: Enter your customization requests

[0581] The user inputs a request for customizing the virtual interviewer using the operation screen of the terminal.

[0582] As an example of input, select evaluation items such as "programming skills" and "knowledge of algorithms."

[0583] The terminal collects the input customization requests and transmits them to the server as a customization request.

[0584] Input: The customization request that the user types into the terminal.

[0585] Output: Customization request data generated by the device

[0586] Step 2: Parsing the customization request

[0587] The server receives a customization request sent from the terminal.

[0588] The server analyzes the received data and extracts parameters to input into the generative AI model.

[0589] Data analysis involves classifying and organizing information based on industry, job type, and assessed skills.

[0590] Input: Customization request sent from the device

[0591] Output: Analysis results to be input into the generative AI model

[0592] Step 3: Generate a virtual interviewer

[0593] The server generates a virtual interviewer using a generative AI model based on the analysis results.

[0594] The generated virtual interviewer has a specific question pattern and evaluation criteria.

[0595] The generated virtual interviewer profile is stored in the server.

[0596] Input: Analysis results (parameters reflecting user customization requests)

[0597] Output: Virtual interviewer profile

[0598] Step 4: Schedule and prepare for the interview

[0599] The user operates the terminal to set an interview date in order to send an interview invitation to the candidate.

[0600] The terminal transmits the set schedule information to the server and prepares for the candidate to log in at the specified date and time.

[0601] The server activates the virtual interviewer at the start time of the interview.

[0602] Input: Interview schedule information

[0603] Output: Sending interview invitations and preparing the system

[0604] Step 5: Start the interview and ask questions

[0605] The server provides the candidate with initial questions through a virtual interviewer.

[0606] The server generates questions based on the evaluation items set by the user.

[0607] The candidate enters their response through the terminal, which is then sent to the server.

[0608] Input: Candidate login and responses to questions

[0609] Output: Candidate response data

[0610] Step 6: Parse the response and generate the next question

[0611] The server analyzes the candidate's responses and generates the next question based on the content in real time.

[0612] For example, if you answer "I have experience with Python and Java," questions like "What specific projects have you used those languages ​​in?" will be generated.

[0613] Input: Candidate response data

[0614] Output: Next question

[0615] Step 7: Gather feedback

[0616] After the interview, the user uses the terminal to enter feedback.

[0617] Feedback may include an evaluation of the interview as a whole and an evaluation of specific questions and responses.

[0618] The terminal sends the feedback to the server.

[0619] Input: User feedback information

[0620] Output: Feedback data

[0621] Step 8: Feedback analysis and optimization

[0622] The server analyzes the collected feedback and interview performance data.

[0623] The server optimizes the virtual interviewer's questioning style and evaluation criteria based on the analysis results.

[0624] For example, if you receive feedback that there are not enough questions to assess logical thinking, add a question such as "Please explain the specific steps you took to solve the problem" to your next interview.

[0625] Input: Feedback data and interview performance data

[0626] Output: Optimized question style and evaluation criteria

[0627] Step 9: Store and analyze the interview data

[0628] The server collects and stores data from all interview sessions.

[0629] The server analyses the collected data and provides the user with insight into the interview process.

[0630] Input: Interview session data

[0631] Output: Interview process insights and reports

[0632] Step 10: Dynamically Adjust System Resources

[0633] The server dynamically adjusts system resources according to the company's adoption volume.

[0634] When multiple interviews are conducted at once, the server automatically increases computing resources and distributes the load to ensure stable terminal operation.

[0635] Input: Change in recruitment volume

[0636] Output: Dynamically adjusted system resources

[0637] (Application example 1)

[0638] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0639] In modern factories, accurately evaluating the capabilities of automated robots and efficiently training them are key challenges. Robots with diverse functions in particular need to have their skills evaluated and improved appropriately, but existing systems make it difficult to customize individual evaluation criteria and training scenarios, making it difficult to ensure an optimal training process.

[0640] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0641] In this invention, the server includes a means for generating a customized virtual trainer based on a user's request, a means for conducting real-time interaction with the trainee device and performing skill evaluation, and a means for collecting and analyzing training data to provide an optimized training experience, thereby enabling accurate evaluation of the robot's capabilities and providing adaptive and efficient training.

[0642] A "user" is an individual or group that operates the system or requests customization.

[0643] A "virtual trainer" is a virtual role model that is generated based on a user's request and performs skill evaluation and training on the trainee device.

[0644] "Training equipment" refers to machinery and equipment that is the subject of training and evaluation, such as robots and automated operating devices used in factories.

[0645] "Real-time interaction" refers to the process in which the virtual trainer interacts directly with the trainee device in real time, exchanging questions and instructions in real time.

[0646] "Skills evaluation" is the act of measuring and evaluating abilities such as the skill to operate the training equipment, reaction speed, and error handling ability.

[0647] "Training Data" means data collected during a training session, including the response, performance, and feedback of the trained device.

[0648] An "optimized training experience" is a training session that is customized to the characteristics of the user and the equipment being trained, and is delivered efficiently and effectively.

[0649] "Feedback" refers to ratings and comments provided by users and those involved in the training, which are used to improve the virtual trainer and optimize the training.

[0650] The "question style" refers to the format or method of questions that the virtual trainer asks the trainee device.

[0651] "Evaluation criteria" are the standards or metrics used to evaluate the trainee's skills.

[0652] The present invention provides a system for effectively training and evaluating robots in factories. An embodiment of the system will be described in detail below.

[0653] The system primarily consists of a user, a terminal, and a server. The user, such as a factory staff member, inputs the characteristics and evaluation criteria of the robot they want to train. The terminal is a computer, tablet, or smartphone operated by the user, and is responsible for collecting information from these and sending it to the server. The server uses a generative AI model to generate a virtual trainer and evaluates the robot's operation and performs training in real time.

[0654] Program processing

[0655] The server performs the process as follows:

[0656] 1. Receive user customization requests

[0657] The user inputs the skills they want to evaluate (e.g., operation skills, reaction speed, error handling ability) through the terminal, and this information is sent to the server.

[0658] 2. Creating a Virtual Trainer

[0659] The server uses the OpenAI API to generate prompts based on the user's requests. For example, the customization request is converted into a prompt, such as "User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, error handling ability." The server then uses the generative AI model to generate a virtual trainer.

[0660] 3. Training commencement and evaluation

[0661] The server activates a virtual trainer, which interacts with the trainee in real time. The virtual trainer presents the robot with a series of questions and tasks, and collects and evaluates its responses. For example, it runs scenarios to evaluate skills such as object recognition, obstacle avoidance, and efficient route planning.

[0662] 4. Data Collection and Optimization

[0663] Data collected during the training session (robot responses, performance data, user feedback, etc.) is stored and analyzed on a server. Based on the results of this analysis, the virtual trainer's questioning style and evaluation criteria are optimized.

[0664] Hardware and software used

[0665] Hardware: Factory robots (e.g., transport robots, assembly robots), user-operated devices (computers, tablets, smartphones)

[0666] Software: OpenAI API, Python program

[0667] Specific examples

[0668] When training a transport robot in a factory, the user inputs evaluation skills using a terminal. For example, if they want to evaluate "operation skills, reaction speed, and error handling ability," this is converted into prompt statements and sent to the server. The virtual trainer presents the robot with a scenario such as, "To evaluate your operation skills, please recognize and transport objects accurately." The robot's behavior is evaluated in real time, and after training is complete, it is optimized based on the feedback.

[0669] Example prompt sentence:

[0670] User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, error handling ability

[0671] Scenario: The virtual trainer sets up an evaluation scenario for the transport robot and records its responses in real time.

[0672] This embodiment allows accurate evaluation of the capabilities of robots in a factory and effective training.

[0673] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0674] Step 1:

[0675] The user inputs the skills they want to evaluate and the characteristics of the robot they are training through their device. For example, the user specifies "modelXYZ" as the robot they want to evaluate for its "operation skills, reaction speed, and error handling ability." This input information is sent from the device to the server.

[0676] Step 2:

[0677] The server generates a prompt based on the received user customization request. The generated prompt is "User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, and error handling ability." This provides the specific data required to generate a virtual trainer.

[0678] Step 3:

[0679] The server generates a virtual trainer using the generative AI model. Specifically, it calls the OpenAI API and provides prompts as input data. The API response generates scripts and scenarios for the virtual trainer, which are then stored on the server.

[0680] Step 4:

[0681] The server activates the virtual trainer to start training. The server sends a training start command to the terminal, and the terminal sends a training start signal to the trained device (robot). At this point, the virtual trainer begins to develop a scenario suitable for the model XYZ robot.

[0682] Step 5:

[0683] A virtual trainer sets up evaluation scenarios for the robot in real time, presenting questions and tasks to the robot, which then perform specific tasks such as object recognition, obstacle avoidance, and efficient route planning. This process generates data on the robot's responses.

[0684] Step 6:

[0685] The server collects the robot's response data generated in real time, including task completion time, number of errors, and performance indicators. This data is stored on the server as training results.

[0686] Step 7:

[0687] The user inputs feedback on the training results through the device. For example, they can provide specific feedback such as, "The robot's reaction speed is appropriate, but its obstacle avoidance is insufficient." This feedback is then sent from the device to the server.

[0688] Step 8:

[0689] The server analyzes user feedback and collected training data to optimize the virtual trainer's questioning style and evaluation criteria, thereby providing a more adaptive and efficient scenario for the next training session.

[0690] Step 9:

[0691] The server updates the optimized virtual trainer profile and makes it available for the next use, thereby achieving continuous training and assessment improvement.

[0692] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0693] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion engine. This system has various functions, such as generating a customized virtual interviewer based on the user's request, conducting real-time Q&A, analyzing and providing interview data, and optimizing question styles based on feedback. Furthermore, by incorporating an emotion engine, the system evaluates the candidate's emotional state, providing more human-like interaction and comprehensive evaluation.

[0694] 1. Interviewer generation and customization

[0695] The user inputs a customization request to generate an interviewer for a specific industry and job type through the device's operation screen. For example, if the goal is to hire a software engineer, the user inputs the skills (programming skills and algorithm knowledge) and characteristics (problem-solving ability and communication ability) they want to evaluate. The device collects this information and sends it to the server as a customization request.

[0696] The server uses a generative AI model to generate a virtual interviewer profile based on the user's input data. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0697] 2. Real-time Q&A

[0698] The user sends an interview invitation to the candidate and schedules the interview. The terminal is prepared so that the candidate can log in at the specified date and time. The server activates the virtual interviewer at the start time of the interview, and also activates the emotion engine.

[0699] The server asks the first question via a virtual interviewer, and the candidate inputs their response via their device. The device then sends this response to the server, where the emotion engine analyzes the candidate's facial expressions, tone of voice, and choice of words to evaluate their emotional state. For example, if a candidate answers, "I have experience with Python and Java," the tone of voice and facial expressions will be used to evaluate whether they are speaking with confidence.

[0700] The server takes into account the analysis results of the emotion engine and generates the next question in real time. The next question is adjusted according to the candidate's emotional state. For example, if the emotion engine determines that the candidate is nervous, the server will present a relaxing question such as "Tell us about your specific project experience."

[0701] This question and answer process is repeated until all questions are answered, and all data collected by the server is recorded.

[0702] 3. Evolving learning functions

[0703] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the server will reflect this in the AI ​​model and ask more detailed questions in the next interview.

[0704] 4. Interview data analysis

[0705] All data from the interview session is collected and stored by the server, including the candidate's responses, evaluation scores, collected sentiment data, and feedback. The server analyzes this data and provides insights to the user, such as average technical skill scores across multiple candidates or trends in specific skill deficiencies, and provides these to the user in the form of a report.

[0706] 5. Providing flexible and scalable solutions

[0707] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the computing resources will be automatically increased. This allows for load balancing to ensure stable operation of terminals.

[0708] Users can customize the settings from their devices for different job types and departments. For example, they can create virtual interviewers for marketing and sales positions, as well as software engineers. The server generates virtual interviewers based on each requirement and helps ensure the interview process goes smoothly.

[0709] The above is an embodiment of the present invention, which enables companies to efficiently and effectively recruit talented personnel by using a virtual interviewer incorporating an emotion engine.

[0710] The processing flow will be explained below.

[0711] 1. Interviewer generation and customization

[0712] Step 1:

[0713] The user inputs a customization request for generating an interviewer corresponding to a specific industry and job type via the operation screen of the terminal.

[0714] Step 2:

[0715] The terminal collects the information input by the user and transmits it to the server as a customization request.

[0716] Step 3:

[0717] The server analyzes the requests received from users and uses a generative AI model to generate a profile for the virtual interviewer, which includes question patterns and evaluation criteria appropriate for the industry and job type.

[0718] Step 4:

[0719] The server stores the generated profile of the virtual interviewer, and the interview preparation is complete.

[0720] 2. Real-time Q&A

[0721] Step 1:

[0722] The user sends interview invitations to candidates and schedules interview dates.

[0723] Step 2:

[0724] The terminal will be ready for the candidate to log into the system at the specified date and time.

[0725] Step 3:

[0726] The server activates the virtual interviewer and the emotion engine in accordance with the start time of the interview.

[0727] Step 4:

[0728] The server asks the first question via a virtual interviewer.

[0729] Step 5:

[0730] The candidate enters the response through a terminal.

[0731] Step 6:

[0732] The terminal transmits the candidate's response to the server.

[0733] Step 7:

[0734] The server analyzes the candidate's responses in real time and generates the next question based on the content, while an emotion engine is initiated to analyze the candidate's facial expressions, tone of voice, choice of words, etc. to evaluate their emotional state.

[0735] Step 8:

[0736] The server considers the results of the emotion engine's evaluation and adjusts the content of questions and pace of the interview depending on the candidate's emotional state. For example, if it determines that the candidate is nervous, it selects questions that will help them relax, such as "Tell us about your specific project experience."

[0737] Step 9:

[0738] The terminal presents the next question received from the server to the candidate.

[0739] Step 10:

[0740] This process is repeated until all questions have been completed.

[0741] 3. Evolving learning functions

[0742] Step 1:

[0743] After the interview, the user enters feedback through the system.

[0744] Step 2:

[0745] The terminal collects feedback from the user and transmits it to the server.

[0746] Step 3:

[0747] The server analyzes the collected feedback and interview performance data. Based on the analysis results, the AI ​​model automatically adjusts the questioning style and evaluation criteria. For example, if the feedback states that "there were not enough questions to evaluate logical thinking," the AI ​​model will add more detailed questions to the next interview.

[0748] 4. Interview data analysis

[0749] Step 1:

[0750] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, emotion data from the emotion engine, and feedback.

[0751] Step 2:

[0752] The server analyzes the collected data and provides the user with insights into the interview process, such as analyzing the average technical skill scores of multiple candidates or trends in deficiencies in specific skills, and presents the results to the user in a report.

[0753] Step 3:

[0754] The terminal displays the analysis results provided by the server to the user.

[0755] 5. Providing flexible and scalable solutions

[0756] Step 1:

[0757] The server dynamically adjusts system resources according to the company's hiring volume. For example, if a large number of interviews are being conducted at once, the server automatically increases computing resources.

[0758] Step 2:

[0759] The terminals perform load balancing to ensure stable operation of the system.

[0760] Step 3:

[0761] Users can customize settings for different job types and departments from their terminals.

[0762] Step 4:

[0763] The server generates an appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[0764] This allows companies to use virtual interviewers with an embedded emotion engine to efficiently and effectively recruit top talent.

[0765] Example 2

[0766] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0767] Conventional interview systems have difficulty efficiently evaluating a large number of candidates at once, and customization for specific industries or job types is difficult. Furthermore, they lack the ability to adjust questions in real time to take into account the candidate's emotional state, resulting in inconsistent interview quality. Furthermore, they lack the learning functionality to appropriately reflect feedback and evolve questioning styles and evaluation criteria.

[0768] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for generating a customized virtual interviewer based on user requests; means for conducting real-time dialogue with candidates and conducting Q&A sessions; means for collecting and analyzing interview data to provide an optimized interview experience; means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results; means for evaluating the candidate's emotional state using an emotion analysis engine and adjusting the content of questions in real time; and means for the server to dynamically adjust resources and perform load balancing when multiple interviews are conducted simultaneously. This simplifies customization for specific industries and job types and enables real-time question adjustment taking into account the candidate's emotional state. Furthermore, the quality of interviews can be improved through efficient analysis of interview data and reflection of feedback.

[0769] A "user" is an entity that interacts with the system to customize a virtual interviewer and coordinate and oversee interviews with candidates.

[0770] "Terminal" means the device used by a user or candidate to access the system, including a PC, smartphone, tablet, etc.

[0771] The "server" is a central processing unit that manages and operates the entire system, generates virtual interviewers, answers questions in real time, analyzes data, and evaluates emotional states.

[0772] A "virtual interviewer" is a virtual presence that is customized based on user requirements using generative AI models to interact with candidates and ask questions in real time.

[0773] An "emotion analysis engine" is a software component that analyzes a candidate's facial expressions, tone of voice, choice of words, etc. to assess their emotional state.

[0774] A "generative AI model" is an artificial intelligence algorithm that generates a profile of a virtual interviewer based on user input data and creates question patterns and evaluation criteria.

[0775] A "customization request" is a requirement or condition provided by a user to generate a virtual interviewer suited to a particular industry or job type.

[0776] "Real-time interaction" refers to the process in which a virtual interviewer asks and responds to candidates in real time.

[0777] "Load balancing" is the process by which a server dynamically adjusts computing resources and distributes them to maintain system stability when multiple interviews are running simultaneously.

[0778] "Feedback" refers to opinions and evaluations provided by users through the system after the interview, and is used to optimize questioning style and evaluation criteria.

[0779] "Questioning style" refers to the format and method of asking questions used by the virtual interviewer.

[0780] "Rating criteria" are the standards or metrics that the virtual interviewer uses to evaluate the candidate's skills and attributes.

[0781] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion analysis engine. This system has various functions, such as generating a customized virtual interviewer based on the user's request, conducting real-time Q&A, analyzing and providing interview data, and optimizing question styles based on feedback. Furthermore, by incorporating an emotion analysis engine, it evaluates the candidate's emotional state, providing a more human-like interaction and comprehensive evaluation.

[0782] First, the user inputs a customization request to generate an interviewer for a specific industry and job type through the device's operation screen. This operation screen is provided on devices such as PCs, tablets, and smartphones. For example, the user might input a request such as, "If I'm looking to hire a software engineer, I'd like to evaluate Python and Java programming skills as the skills I want to assess, and problem-solving ability and communication ability as the attributes." The device receives this request and sends it to the server.

[0783] The server uses a generative AI model to generate a virtual interviewer profile based on the user's customization request. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0784] As a specific prompt, the server sends the following request to the generative AI model: "Please generate a virtual interviewer to conduct job interviews for software engineers. The skills I want to evaluate are programming skills in Python and Java. I also want to evaluate problem-solving ability and communication ability as traits."

[0785] After preparing for the interview, the user uses the virtual interviewer's profile to send an interview invitation to the candidate. The device prepares the necessary equipment so that the candidate can log in at the specified date and time. When the interview starts, the server activates the virtual interviewer and the sentiment analysis engine as well. This allows for real-time question and answer sessions.

[0786] The server asks the candidate an initial question via a virtual interviewer, and the candidate enters their response via their device. The device then sends the response data to the server, where an emotion analysis engine analyzes the candidate's facial expressions, tone of voice, and word choice in real time to assess their emotional state. For example, if a candidate answers, "I have experience with Python and Java," the server adjusts the next question based on the results of the emotion analysis engine. If the candidate is nervous, it can ask questions that will help them relax.

[0787] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the AI ​​model will be informed to add more detailed questions in the next interview.

[0788] Additionally, all data collected during the interview session is stored and analyzed by the server, including the candidate's responses, evaluation scores, collected sentiment data, feedback, etc. The server uses this data to provide insights to the user and provides reports analyzing trends in specific skill deficiencies and average technical skill scores across multiple candidates.

[0789] The server also dynamically adjusts resources and distributes the load to ensure stable system operation even when multiple interviews are conducted at once. This allows terminals to operate stably and multiple interviews to proceed smoothly. This allows companies to efficiently and effectively recruit top talent using virtual interviewers equipped with an emotion analysis engine.

[0790] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0791] Step 1:

[0792] The user inputs a customization request to generate a virtual interviewer for a specific industry and job type through the device's operation screen. For example, for the purpose of hiring a software engineer, a specific request could be input, such as "The skills I want to evaluate are Python and Java programming skills, and the characteristics I want to evaluate are problem-solving ability and communication ability." The input data is sent from the device to the server as a customization request. Input: Industry, Job Type, Evaluation Skills, Evaluation Characteristics | Output: Customization Request

[0793] Step 2:

[0794] The terminal formats the customization request entered by the user and sends it to the server via the network. A specific prompt would be, "Generate a virtual interviewer to conduct a software engineer job interview. The skills we want to evaluate are programming skills in Python and Java. We also want to evaluate problem-solving ability and communication ability as traits." Input: Customization request | Output: Prompt

[0795] Step 3:

[0796] The server receives the customization request and uses the generative AI model to generate a virtual interviewer profile. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is stored in a database. Input: Prompt sentence | Output: Virtual interviewer profile

[0797] Step 4:

[0798] The user uses the generated virtual interviewer profile to send interview invitations to candidates and arrange interview dates. The terminal prepares the candidate for login and prepares the necessary interface. Input: Virtual interviewer profile | Output: Interview invitation

[0799] Step 5:

[0800] The server activates the virtual interviewer at the start time of the interview. The emotion engine is also activated and ready. The server generates the initial questions and asks them to the candidate. Input: Interview invitation | Output: Initial questions

[0801] Step 6:

[0802] Candidates input their answers through their devices. The devices then send the answers to the server, which then uses an emotion engine to analyze the answers. For example, if a candidate answers "I have experience with Python and Java," the server evaluates whether they are speaking with confidence based on their tone of voice and facial expressions. Input: Candidate's answers | Output: Analysis data

[0803] Step 7:

[0804] The server generates the next question based on the results of the sentiment analysis engine. Depending on the candidate's level of nervousness, the server adjusts the questions to present a more relaxed response. The question and answer process is repeated until all questions have been completed. Input: Analysis data | Output: Next question

[0805] Step 8:

[0806] After the interview, the user writes down their feedback, which is then sent to the server. The server analyzes the feedback and optimizes the question style and evaluation criteria for the next interview. Input: Feedback | Output: Optimized question style

[0807] Step 9:

[0808] The server collects and stores data from the entire interview session, performs analysis, and provides insights to the user, including average technical skill scores and trends in specific skill deficiencies. Input: Interview data, analysis data | Output: Report

[0809] Step 10:

[0810] The server dynamically adjusts resources according to the company's recruitment volume. When multiple interviews are conducted simultaneously, the server increases computing resources and distributes the load. This ensures stable system operation. Input: Interview schedule | Output: Dynamic resource adjustment

[0811] (Application example 2)

[0812] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0813] Conventional virtual interviewer systems have difficulty taking into account emotional changes during interviews with candidates, resulting in a uniform interview experience. Furthermore, the interviewer's Q&A and evaluations are fixed, resulting in insufficient optimization of the interview. Furthermore, there is insufficient automation of interview data analysis and feedback, which can lead to delays in the evolution of interviewer training and evaluation criteria.

[0814] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0815] In this invention, the server includes means for generating a customized virtual interviewer based on user requests, means for conducting real-time dialogue with candidates and conducting Q&A sessions, means for collecting and analyzing interview data to provide an optimized interview experience, means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results, means for analyzing emotional evaluations, and means for adjusting questions based on the analysis results. This enables interactive interviews that take into account the candidate's emotional state, resulting in a high-quality interview experience and efficient talent evaluation.

[0816] "User" refers to an individual or organization that uses this system.

[0817] A "virtual interviewer" refers to a program that uses generative AI models and emotion engines to mimic the role of a real-life interviewer.

[0818] "Real-time interaction" refers to simultaneous interaction between the interviewer and candidate.

[0819] "Question and answer session" refers to the process in which the interviewer asks questions and the candidate answers them.

[0820] "Interview data" refers to information obtained during an interview, such as the candidate's responses, facial expressions, and tone of voice.

[0821] "Analysis" refers to the process of evaluating the collected interview data and deriving meaningful results.

[0822] An "optimized interview experience" refers to an interview process that is tailored to the candidate's emotional state and reactions.

[0823] "Feedback" refers to the evaluation and opinions provided by the user after the interview is completed.

[0824] "Questioning style" refers to the format or pattern of questions used by the interviewer.

[0825] "Evaluation criteria" refers to the standards or criteria for evaluating a candidate's responses or performance.

[0826] "Emotional assessment" refers to the process of analyzing a candidate's facial expressions and tone of voice to determine their emotional state.

[0827] "Analysis results" refer to the specific conclusions or findings obtained after conducting an emotion assessment.

[0828] "Question tailoring" refers to the process of changing or selecting the next question based on the candidate's emotional state.

[0829] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion engine. This system generates a customized virtual interviewer based on the user's request and has functions such as real-time question and answer sessions, analysis and provision of interview data, and optimization of question styles based on feedback. In addition, by incorporating an emotion engine, it evaluates the candidate's emotional state and provides human-like interaction and comprehensive evaluation.

[0830] System Configuration

[0831] 1. User:

[0832] The user inputs a customization request for generating an interviewer corresponding to a specific industry and job type through the operation screen of the terminal.

[0833] 2. Terminal:

[0834] The terminal collects customization requests from users and sends them to a server. Wearable devices such as smart glasses are used for training and interviewing new operators.

[0835] 3. Server:

[0836] The server uses a generative AI model based on the user's input data to generate a virtual interviewer profile, which includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0837] Real-time Q&A

[0838] The server activates the virtual interviewer at the start of the interview, and also activates the emotion engine. The server asks the first question through the virtual interviewer, and the candidate inputs their response through their terminal. The terminal sends this response to the server, and the emotion engine analyzes the candidate's facial expressions, tone of voice, choice of words, etc. to evaluate their emotional state.

[0839] For example, if a candidate answers, "I have experience operating machines," the emotion engine evaluates whether they are speaking with confidence based on their tone of voice and facial expressions. The server takes this analysis into account and generates the next question in real time. For example, if the emotion engine determines that the candidate is nervous, the server will present a question that will help them relax, such as, "Tell us about your specific project experience."

[0840] Learning Function

[0841] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the server will reflect this in the AI ​​model and ask more detailed questions in the next interview.

[0842] Data analysis and provision

[0843] All data from the interview session is collected and stored by the server, including the candidate's responses, evaluation scores, collected sentiment data, and feedback. The server analyzes this data and provides insights to the user. For example, it analyzes the average technical skill scores of multiple candidates and trends in specific skill deficiencies, and provides this to the user as a report.

[0844] Providing flexible and scalable solutions

[0845] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the server can automatically increase computing resources and distribute the load to ensure stable operation of the terminals.

[0846] Examples and prompts

[0847] For example, if a new factory operator answers, "I would like to learn how to operate a new machine," during training for new employees, the virtual interviewer will analyze their tone of voice and facial expression, and if it determines that they are nervous, it will put them at ease by asking them the next question, "Please explain the specific operating procedures."

[0848] Example prompt sentence:

[0849] You are a virtual training assistant helping train new employees in a factory. Based on the responses provided by the candidate, you use an emotion engine to analyze their emotional state and tailor the next questions you ask.

[0850] This enables interactive interviews that take into account the candidate's emotional state, resulting in a high-quality interview experience and efficient talent evaluation.

[0851] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0852] Step 1:

[0853] The user operates the device to input customization requests to generate a virtual interviewer for a specific industry and job type. The data input by the user includes the skills (e.g., programming skills, knowledge of machine operations) and characteristics (e.g., problem-solving ability, communication ability) to be evaluated. The input data is sent to the server in JSON format.

[0854] Step 2:

[0855] The device collects customization requests from the user and sends them to the server. The data sent as the customization request includes the user's input information. Upon receiving this data, the server uses a generative AI model to generate a profile for the virtual interviewer. The profile includes question patterns and evaluation criteria appropriate for the industry and job type.

[0856] Step 3:

[0857] The server saves the generated virtual interviewer profile and prepares for the interview. The profile saves specific questions and evaluation criteria in template format. The saved profile is used in subsequent processing.

[0858] Step 4:

[0859] After the interview date is arranged, the terminal sends an interview invitation to the candidate, and prepares the candidate so that they can log in at the specified date and time. The server verifies the login information and activates the virtual interviewer at the start time of the interview.

[0860] Step 5:

[0861] When the interview begins, the server asks the first question via the virtual interviewer, and the candidate enters their response via the terminal. The entered response data is sent in text format to the server, which then receives the candidate's response and analyzes the data using a natural language processing engine.

[0862] Step 6:

[0863] The server receives the candidate's responses and uses an emotion engine to analyze the candidate's facial expressions, tone of voice, and word choice. Based on the analysis results, the server evaluates the candidate's emotional state (e.g., nervous, confident, confused). The evaluation data is used to generate the next questions in real time.

[0864] Step 7:

[0865] Taking into account the analysis results of the emotion engine, the server generates the next question in real time. For example, if the server determines that the candidate is nervous, it will present a question that will help them relax, such as "Tell us about your specific project experience." The generated question is displayed to the candidate via their device.

[0866] Step 8:

[0867] The question and answer process is repeated until all questions are completed. The server records all collected data and stores all data from the interview session, including the candidate's responses, evaluation scores, collected emotional data, and feedback.

[0868] Step 9:

[0869] After the interview, the user enters feedback through the system. The terminal collects the feedback and sends it to the server. The server analyzes the collected feedback and optimizes the question style and evaluation criteria for the next interview, thereby improving the accuracy and effectiveness of the virtual interviewer.

[0870] Step 10:

[0871] The server generates an interview report based on the analysis results and provides it to the user. The report includes the average technical skill scores of multiple candidates, trends in specific skill deficiencies, and an overall evaluation of the interview. Users can use this report to optimize their recruitment activities.

[0872] The above are the specific processing steps for implementing the present invention, which enable interactive interviews that take into account the candidate's emotional state, thereby achieving a high-quality interview experience and efficient talent evaluation.

[0873] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0874] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0875] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0876] [Third embodiment]

[0877] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0878] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0879] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0880] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0881] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0882] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0883] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0884] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0885] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0886] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0887] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0888] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0889] The present invention will be described below as a specific method for implementing the invention, with reference to the operation of a system using a user, a terminal, and a server. The invention is implemented based on the following configuration.

[0890] 1. Interviewer generation and customization

[0891] 1-1: Receive customization requests

[0892] Users use the system to submit customization requests to generate interviewers for specific industries and job types. For example, if the user is looking to hire a software engineer, the user inputs the skills (e.g., programming skills and algorithm knowledge) and characteristics (e.g., problem-solving ability and communication ability) they want to evaluate on the device's operation screen. The device then collects this information and sends it to the server as a customization request.

[0893] 1-2: Generate interviewers

[0894] The server analyzes the customization request sent by the user and generates a virtual interviewer using a generative AI model based on the analysis. The virtual interviewer has question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[0895] 2. Real-time Q&A

[0896] 2-1: Start of the interview

[0897] The user sends an interview invitation to the candidate and schedules the interview. The terminal is prepared so that the candidate can log in to the system at the specified date and time. The server activates the virtual interviewer at the start time of the interview.

[0898] 2-2: Providing questions and recording responses

[0899] The server asks the candidate an initial question via a virtual interviewer. For example, "Tell me about your programming experience." When the candidate enters their response via their terminal, it is sent to the server. The server analyzes the candidate's response and generates the next question in real time based on the content. For example, if the candidate answers "I have experience with Python and Java," the server generates the next question in real time: "In what specific projects have you used those languages?" This allows for a detailed evaluation of the candidate's skills and aptitude.

[0900] 3. Evolving learning functions

[0901] 3-1: Collecting and sending feedback

[0902] After the interview, the user enters feedback through the system, including an evaluation of the overall interview and evaluation of specific questions and responses. The terminal collects this feedback and sends it to the server.

[0903] 3-2: Optimizing question style and evaluation criteria

[0904] The server analyzes the collected feedback and interview performance data. For example, if feedback indicates that there are not enough questions to evaluate the candidate's logical thinking, the system can use this information to adjust the questioning style and add a question such as "Please explain the specific steps you took to solve the problem" to the next interview. In this way, the virtual interviewer can continuously evolve and provide more effective interviews.

[0905] 4. Interview data analysis

[0906] 4-1: Collection of interview data

[0907] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback.

[0908] 4-2: Analyzing data and providing insights

[0909] The server analyzes the collected data and provides the user with insights into the interview process, such as analyzing the average technical skill scores of multiple candidates or specific skill deficiencies, and generates a report that is provided to the user, allowing the user to gain specific insights to further optimize the hiring process.

[0910] 5. Providing flexible and scalable solutions

[0911] 5-1: Scaling up

[0912] The server dynamically adjusts system resources according to the company's hiring volume. For example, if a large number of interviews are being conducted at once, the necessary computing resources will be automatically increased. The server also distributes the load to ensure stable operation of terminals.

[0913] 5-2: Meeting different hiring needs

[0914] Users can use their devices to customize settings for different job types and departments. For example, they can set up the system to generate virtual interviewers that are optimal for each job type, so that it can accommodate not only software engineers but also marketing and sales positions. The server generates the appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[0915] The above is an embodiment of the present invention, which enables companies to recruit talented personnel efficiently and effectively.

[0916] The processing flow will be explained below.

[0917] 1. Interviewer generation and customization

[0918] Step 1:

[0919] The user inputs the industry, job type, and skills and characteristics they want to evaluate through the device's operation screen.

[0920] Step 2:

[0921] The terminal collects input information from the user and transmits it to the server as a customization request.

[0922] Step 3:

[0923] The server analyzes the received request and uses a generative AI model to generate a profile for the virtual interviewer, which includes question patterns and evaluation criteria appropriate for the industry and job type.

[0924] Step 4:

[0925] The server stores the generated virtual interviewer profile and the interview is ready.

[0926] 2. Real-time Q&A

[0927] Step 1:

[0928] The user sends interview invitations to candidates and schedules interviews.

[0929] Step 2:

[0930] The terminal will be ready for the candidate to log into the system at the appointed date and time.

[0931] Step 3:

[0932] The server activates the virtual interviewer at the start time of the interview.

[0933] Step 4:

[0934] The server asks the candidate an initial question via a virtual interviewer.

[0935] Step 5:

[0936] The candidate enters their response through a terminal.

[0937] Step 6:

[0938] The terminal sends the candidate's response to the server.

[0939] Step 7:

[0940] The server analyzes the candidate's responses and generates subsequent questions in real time based on their content. For example, if a question about technical skills follows, the server generates a question such as, "What specific projects have you used those languages ​​in?"

[0941] Step 8:

[0942] The server generates the next question and sends it to the terminal, which then presents it to the candidate.

[0943] Step 9:

[0944] This process is repeated until all questions have been completed.

[0945] 3. Evolving learning functions

[0946] Step 1:

[0947] After the interview, the user enters feedback through the system.

[0948] Step 2:

[0949] The device collects the feedback and sends it to the server.

[0950] Step 3:

[0951] The server analyzes the collected feedback and interview performance data. Based on the analysis results, the question style and evaluation criteria are automatically adjusted according to the AI ​​model. For example, if feedback indicates that there are a lack of questions that evaluate logical thinking, relevant question patterns are added.

[0952] 4. Interview data analysis

[0953] Step 1:

[0954] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback.

[0955] Step 2:

[0956] The server analyzes the collected data and provides the user with insight into the interview process, such as average technical skill scores or identifying skill gaps, and generates a report of the results.

[0957] Step 3:

[0958] The terminal displays the analysis results provided by the server to the user.

[0959] 5. Providing flexible and scalable solutions

[0960] Step 1:

[0961] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the computing resources will be automatically increased.

[0962] Step 2:

[0963] Terminals distribute the load to ensure stable operation of the system.

[0964] Step 3:

[0965] Users can customize settings for different job types or departments from their terminals.

[0966] Step 4:

[0967] The server generates an appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[0968] The above are the specific processing steps of the present invention, and this flow enables companies to recruit talented personnel efficiently and effectively.

[0969] Example 1

[0970] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0971] In the modern recruitment process, there are many challenges in candidate evaluation and interview efficiency. In particular, evaluations are often influenced by the interviewer's subjectivity, preparing for and conducting interviews requires a great deal of time and resources, and continuous improvement based on feedback is difficult. There is a need to solve these challenges and provide a more effective and efficient interview process.

[0972] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0973] In this invention, the server includes a means for generating a customized virtual interviewer based on user requests, a means for conducting real-time dialogue with candidates and conducting Q&A sessions, and a means for collecting and analyzing interview data to provide an optimized interview experience. This streamlines the interview process and enables fair and objective candidate evaluation. Furthermore, the virtual interviewer's questioning style and evaluation criteria can be automatically learned using user feedback and interview results, enabling continuous improvement. Furthermore, because the customized virtual interviewer is generated using a generative AI model, it is easy to create a virtual interviewer with question patterns and evaluation criteria suited to specific industries and job types.

[0974] A "user request" is a request by a user to customize the virtual interviewer based on a particular industry, job type, and assessment skills when generating the virtual interviewer.

[0975] A "customized virtual interviewer" is a virtual interviewer with question patterns and evaluation criteria suited to the needs of a specific industry or job type, according to the user's request.

[0976] A "generative AI model" is an artificial intelligence model that analyzes user requirements and generates a customized virtual interviewer based on those requirements.

[0977] "Real-time dialogue" refers to the process of an interview in which the candidate and virtual interviewer interactively exchange questions and answers, with immediate responses and feedback.

[0978] A "question and answer" is a series of interactions in which a virtual interviewer asks a candidate questions and the candidate responds.

[0979] "Interview data" refers to data collected during an interview, including information such as the candidate's responses, evaluation scores, and feedback.

[0980] An "optimized interview experience" refers to an interview process that improves the quality and efficiency of interviews by analyzing interview data and utilizing feedback.

[0981] "Feedback" refers to the evaluation of the entire interview provided by the user after the interview is completed, as well as evaluation comments on specific questions and answers.

[0982] "Questioning style" is a concept that refers to the format and manner in which a virtual interviewer asks questions to a candidate.

[0983] "Evaluation criteria" are indicators or standards that the virtual interviewer uses to evaluate the candidate's responses, and are set based on the user's requests.

[0984] A "profile" refers to a generated set of information about a virtual interviewer, including data related to interview preparation.

[0985] "Analysis" is the process of analyzing the collected interview data and generating insights and recommendations based on the results.

[0986] The present invention is an interview support system implemented using a user, a terminal, and a server, and a specific implementation method thereof will be described below. Implementing the invention requires cooperation between hardware and software.

[0987] 1. Interviewer generation and customization

[0988] To generate a virtual interviewer for a specific industry and job type, the user inputs customization requests using the device's operation screen. Examples of input include evaluation items such as "programming skills," "algorithm knowledge," "problem-solving ability," and "communication ability." The device collects this input information and sends it to the server as a customization request.

[0989] The server analyzes the received customization request and uses a generative AI model to generate a customized virtual interviewer with question patterns and evaluation criteria suited to the specific industry and job type. The generated virtual interviewer profile is stored on the server and is ready for the interview.

[0990] 2. Real-time Q&A

[0991] The user sends an interview invitation to the candidate from the terminal and arranges the interview date. The terminal makes the necessary preparations so that the candidate can log in to the system at the specified date and time. The server activates the virtual interviewer at the start time of the interview.

[0992] When the interview begins, the server provides the candidate with the first question through the virtual interviewer. A specific example question is, "Tell me about your programming experience." The candidate enters their response through their terminal, which is then sent to the server. The server analyzes the candidate's response and generates the next question in real time based on the content. For example, if the candidate answers, "I have experience with Python and Java," the server generates a question such as, "In what specific projects have you used those languages?"

[0993] 3. Evolving learning functions

[0994] After the interview, the user enters feedback through the system. The feedback includes an evaluation of the entire interview and evaluation of specific questions and answers. The terminal collects this feedback and sends it to the server.

[0995] The server analyzes the collected feedback and interview performance data to optimize the virtual interviewer's questioning style and evaluation criteria. For example, if feedback indicates that there are not enough questions to evaluate the candidate's logical thinking, the server will add a question such as "Please explain the specific steps you took to solve the problem" to the next interview. In this way, the virtual interviewer continuously evolves and provides more effective interviews.

[0996] 4. Interview data analysis

[0997] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback. The server analyzes the collected data and provides users with insights into the interview process. For example, the average technical skill scores of multiple candidates and specific skill deficiencies are analyzed and provided to users as a report. This allows users to gain specific insights to further optimize the hiring process.

[0998] 5. Providing flexible and scalable solutions

[0999] The server dynamically adjusts system resources according to the company's hiring volume. For example, when conducting a large number of interviews at once, it automatically increases the necessary computing resources and distributes the load to ensure stable terminal operation. Users can also use their terminals to customize settings for different job types and departments. For example, it is possible to set up the system to generate virtual interviewers optimized for each job type, so that it can accommodate not only software engineers but also marketing and sales positions. The server generates appropriate virtual interviewers based on each request and supports the smooth conduct of interviews.

[1000] Specific examples

[1001] An example prompt might be, "What skills are necessary to generate software engineer interviewers?", to which the system generates specific answers such as "programming skills, knowledge of algorithms, problem-solving ability, and communication skills."

[1002] By following the above procedure, the present invention can be implemented, enabling companies to recruit talented personnel efficiently and effectively.

[1003] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1004] Step 1: Enter your customization requests

[1005] The user inputs a request for customizing the virtual interviewer using the operation screen of the terminal.

[1006] As an example of input, select evaluation items such as "programming skills" and "knowledge of algorithms."

[1007] The terminal collects the input customization requests and transmits them to the server as a customization request.

[1008] Input: The customization request that the user types into the terminal.

[1009] Output: Customization request data generated by the device

[1010] Step 2: Parsing the customization request

[1011] The server receives a customization request sent from the terminal.

[1012] The server analyzes the received data and extracts parameters to input into the generative AI model.

[1013] Data analysis involves classifying and organizing information based on industry, job type, and assessed skills.

[1014] Input: Customization request sent from the device

[1015] Output: Analysis results to be input into the generative AI model

[1016] Step 3: Generate a virtual interviewer

[1017] The server generates a virtual interviewer using a generative AI model based on the analysis results.

[1018] The generated virtual interviewer has a specific question pattern and evaluation criteria.

[1019] The generated virtual interviewer profile is stored in the server.

[1020] Input: Analysis results (parameters reflecting user customization requests)

[1021] Output: Virtual interviewer profile

[1022] Step 4: Schedule and prepare for the interview

[1023] The user operates the terminal to set an interview date in order to send an interview invitation to the candidate.

[1024] The terminal transmits the set schedule information to the server and prepares for the candidate to log in at the specified date and time.

[1025] The server activates the virtual interviewer at the start time of the interview.

[1026] Input: Interview schedule information

[1027] Output: Sending interview invitations and preparing the system

[1028] Step 5: Start the interview and ask questions

[1029] The server provides the candidate with initial questions through a virtual interviewer.

[1030] The server generates questions based on the evaluation items set by the user.

[1031] The candidate enters their response through the terminal, which is then sent to the server.

[1032] Input: Candidate login and responses to questions

[1033] Output: Candidate response data

[1034] Step 6: Parse the response and generate the next question

[1035] The server analyzes the candidate's responses and generates the next question based on the content in real time.

[1036] For example, if you answer "I have experience with Python and Java," questions like "What specific projects have you used those languages ​​in?" will be generated.

[1037] Input: Candidate response data

[1038] Output: Next question

[1039] Step 7: Gather feedback

[1040] After the interview, the user uses the terminal to enter feedback.

[1041] Feedback may include an evaluation of the interview as a whole and an evaluation of specific questions and responses.

[1042] The terminal sends the feedback to the server.

[1043] Input: User feedback information

[1044] Output: Feedback data

[1045] Step 8: Feedback analysis and optimization

[1046] The server analyzes the collected feedback and interview performance data.

[1047] The server optimizes the virtual interviewer's questioning style and evaluation criteria based on the analysis results.

[1048] For example, if you receive feedback that there are not enough questions to assess logical thinking, add a question such as "Please explain the specific steps you took to solve the problem" to your next interview.

[1049] Input: Feedback data and interview performance data

[1050] Output: Optimized question style and evaluation criteria

[1051] Step 9: Store and analyze the interview data

[1052] The server collects and stores data from all interview sessions.

[1053] The server analyses the collected data and provides the user with insight into the interview process.

[1054] Input: Interview session data

[1055] Output: Interview process insights and reports

[1056] Step 10: Dynamically Adjust System Resources

[1057] The server dynamically adjusts system resources according to the company's adoption volume.

[1058] When multiple interviews are conducted at once, the server automatically increases computing resources and distributes the load to ensure stable terminal operation.

[1059] Input: Change in recruitment volume

[1060] Output: Dynamically adjusted system resources

[1061] (Application example 1)

[1062] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1063] In modern factories, accurately evaluating the capabilities of automated robots and efficiently training them are key challenges. Robots with diverse functions in particular need to have their skills evaluated and improved appropriately, but existing systems make it difficult to customize individual evaluation criteria and training scenarios, making it difficult to ensure an optimal training process.

[1064] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1065] In this invention, the server includes a means for generating a customized virtual trainer based on a user's request, a means for conducting real-time interaction with the trainee device and performing skill evaluation, and a means for collecting and analyzing training data to provide an optimized training experience, thereby enabling accurate evaluation of the robot's capabilities and providing adaptive and efficient training.

[1066] A "user" is an individual or group that operates the system or requests customization.

[1067] A "virtual trainer" is a virtual role model that is generated based on a user's request and performs skill evaluation and training on the trainee device.

[1068] "Training equipment" refers to machinery and equipment that is the subject of training and evaluation, such as robots and automated operating devices used in factories.

[1069] "Real-time interaction" refers to the process in which the virtual trainer interacts directly with the trainee device in real time, exchanging questions and instructions in real time.

[1070] "Skills evaluation" is the act of measuring and evaluating abilities such as the skill to operate the training equipment, reaction speed, and error handling ability.

[1071] "Training Data" means data collected during a training session, including the response, performance, and feedback of the trained device.

[1072] An "optimized training experience" is a training session that is customized to the characteristics of the user and the equipment being trained, and is delivered efficiently and effectively.

[1073] "Feedback" refers to ratings and comments provided by users and those involved in the training, which are used to improve the virtual trainer and optimize the training.

[1074] The "question style" refers to the format or method of questions that the virtual trainer asks the trainee device.

[1075] "Evaluation criteria" are the standards or metrics used to evaluate the trainee's skills.

[1076] The present invention provides a system for effectively training and evaluating robots in factories. An embodiment of the system will be described in detail below.

[1077] The system primarily consists of a user, a terminal, and a server. The user, such as a factory staff member, inputs the characteristics and evaluation criteria of the robot they want to train. The terminal is a computer, tablet, or smartphone operated by the user, and is responsible for collecting information from these and sending it to the server. The server uses a generative AI model to generate a virtual trainer and evaluates the robot's operation and performs training in real time.

[1078] Program processing

[1079] The server performs the process as follows:

[1080] 1. Receive user customization requests

[1081] The user inputs the skills they want to evaluate (e.g., operation skills, reaction speed, error handling ability) through the terminal, and this information is sent to the server.

[1082] 2. Creating a Virtual Trainer

[1083] The server uses the OpenAI API to generate prompts based on the user's requests. For example, the customization request is converted into a prompt, such as "User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, error handling ability." The server then uses the generative AI model to generate a virtual trainer.

[1084] 3. Training commencement and evaluation

[1085] The server activates a virtual trainer, which interacts with the trainee in real time. The virtual trainer presents the robot with a series of questions and tasks, and collects and evaluates its responses. For example, it runs scenarios to evaluate skills such as object recognition, obstacle avoidance, and efficient route planning.

[1086] 4. Data Collection and Optimization

[1087] Data collected during the training session (robot responses, performance data, user feedback, etc.) is stored and analyzed on a server. Based on the results of this analysis, the virtual trainer's questioning style and evaluation criteria are optimized.

[1088] Hardware and software used

[1089] Hardware: Factory robots (e.g., transport robots, assembly robots), user-operated devices (computers, tablets, smartphones)

[1090] Software: OpenAI API, Python program

[1091] Specific examples

[1092] When training a transport robot in a factory, the user inputs evaluation skills using a terminal. For example, if they want to evaluate "operation skills, reaction speed, and error handling ability," this is converted into prompt statements and sent to the server. The virtual trainer presents the robot with a scenario such as, "To evaluate your operation skills, please recognize and transport objects accurately." The robot's behavior is evaluated in real time, and after training is complete, it is optimized based on the feedback.

[1093] Example prompt sentence:

[1094] User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, error handling ability

[1095] Scenario: The virtual trainer sets up an evaluation scenario for the transport robot and records its responses in real time.

[1096] This embodiment allows accurate evaluation of the capabilities of robots in a factory and effective training.

[1097] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1098] Step 1:

[1099] The user inputs the skills they want to evaluate and the characteristics of the robot they are training through their device. For example, the user specifies "modelXYZ" as the robot they want to evaluate for its "operation skills, reaction speed, and error handling ability." This input information is sent from the device to the server.

[1100] Step 2:

[1101] The server generates a prompt based on the received user customization request. The generated prompt is "User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, and error handling ability." This provides the specific data required to generate a virtual trainer.

[1102] Step 3:

[1103] The server generates a virtual trainer using the generative AI model. Specifically, it calls the OpenAI API and provides prompts as input data. The API response generates scripts and scenarios for the virtual trainer, which are then stored on the server.

[1104] Step 4:

[1105] The server activates the virtual trainer to start training. The server sends a training start command to the terminal, and the terminal sends a training start signal to the trained device (robot). At this point, the virtual trainer begins to develop a scenario suitable for the model XYZ robot.

[1106] Step 5:

[1107] A virtual trainer sets up evaluation scenarios for the robot in real time, presenting questions and tasks to the robot, which then perform specific tasks such as object recognition, obstacle avoidance, and efficient route planning. This process generates data on the robot's responses.

[1108] Step 6:

[1109] The server collects the robot's response data generated in real time, including task completion time, number of errors, and performance indicators. This data is stored on the server as training results.

[1110] Step 7:

[1111] The user inputs feedback on the training results through the device. For example, they can provide specific feedback such as, "The robot's reaction speed is appropriate, but its obstacle avoidance is insufficient." This feedback is then sent from the device to the server.

[1112] Step 8:

[1113] The server analyzes user feedback and collected training data to optimize the virtual trainer's questioning style and evaluation criteria, thereby providing a more adaptive and efficient scenario for the next training session.

[1114] Step 9:

[1115] The server updates the optimized virtual trainer profile and makes it available for the next use, thereby achieving continuous training and assessment improvement.

[1116] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1117] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion engine. This system has various functions, such as generating a customized virtual interviewer based on the user's request, conducting real-time Q&A, analyzing and providing interview data, and optimizing question styles based on feedback. Furthermore, by incorporating an emotion engine, the system evaluates the candidate's emotional state, providing more human-like interaction and comprehensive evaluation.

[1118] 1. Interviewer generation and customization

[1119] The user inputs a customization request to generate an interviewer for a specific industry and job type through the device's operation screen. For example, if the goal is to hire a software engineer, the user inputs the skills (programming skills and algorithm knowledge) and characteristics (problem-solving ability and communication ability) they want to evaluate. The device collects this information and sends it to the server as a customization request.

[1120] The server uses a generative AI model to generate a virtual interviewer profile based on the user's input data. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[1121] 2. Real-time Q&A

[1122] The user sends an interview invitation to the candidate and schedules the interview. The terminal is prepared so that the candidate can log in at the specified date and time. The server activates the virtual interviewer at the start time of the interview, and also activates the emotion engine.

[1123] The server asks the first question via a virtual interviewer, and the candidate inputs their response via their device. The device then sends this response to the server, where the emotion engine analyzes the candidate's facial expressions, tone of voice, and choice of words to evaluate their emotional state. For example, if a candidate answers, "I have experience with Python and Java," the tone of voice and facial expressions will be used to evaluate whether they are speaking with confidence.

[1124] The server takes into account the analysis results of the emotion engine and generates the next question in real time. The next question is adjusted according to the candidate's emotional state. For example, if the emotion engine determines that the candidate is nervous, the server will present a relaxing question such as "Tell us about your specific project experience."

[1125] This question and answer process is repeated until all questions are answered, and all data collected by the server is recorded.

[1126] 3. Evolving learning functions

[1127] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the server will reflect this in the AI ​​model and ask more detailed questions in the next interview.

[1128] 4. Interview data analysis

[1129] All data from the interview session is collected and stored by the server, including the candidate's responses, evaluation scores, collected sentiment data, and feedback. The server analyzes this data and provides insights to the user, such as average technical skill scores across multiple candidates or trends in specific skill deficiencies, and provides these to the user in the form of a report.

[1130] 5. Providing flexible and scalable solutions

[1131] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the computing resources will be automatically increased. This allows for load balancing to ensure stable operation of terminals.

[1132] Users can customize the settings from their devices for different job types and departments. For example, they can create virtual interviewers for marketing and sales positions, as well as software engineers. The server generates virtual interviewers based on each requirement and helps ensure the interview process goes smoothly.

[1133] The above is an embodiment of the present invention, which enables companies to efficiently and effectively recruit talented personnel by using a virtual interviewer incorporating an emotion engine.

[1134] The processing flow will be explained below.

[1135] 1. Interviewer generation and customization

[1136] Step 1:

[1137] The user inputs a customization request for generating an interviewer corresponding to a specific industry and job type via the operation screen of the terminal.

[1138] Step 2:

[1139] The terminal collects the information input by the user and transmits it to the server as a customization request.

[1140] Step 3:

[1141] The server analyzes the requests received from users and uses a generative AI model to generate a profile for the virtual interviewer, which includes question patterns and evaluation criteria appropriate for the industry and job type.

[1142] Step 4:

[1143] The server stores the generated profile of the virtual interviewer, and the interview preparation is complete.

[1144] 2. Real-time Q&A

[1145] Step 1:

[1146] The user sends interview invitations to candidates and schedules interview dates.

[1147] Step 2:

[1148] The terminal will be ready for the candidate to log into the system at the specified date and time.

[1149] Step 3:

[1150] The server activates the virtual interviewer and the emotion engine in accordance with the start time of the interview.

[1151] Step 4:

[1152] The server asks the first question via a virtual interviewer.

[1153] Step 5:

[1154] The candidate enters the response through a terminal.

[1155] Step 6:

[1156] The terminal transmits the candidate's response to the server.

[1157] Step 7:

[1158] The server analyzes the candidate's responses in real time and generates the next question based on the content, while an emotion engine is initiated to analyze the candidate's facial expressions, tone of voice, choice of words, etc. to evaluate their emotional state.

[1159] Step 8:

[1160] The server considers the results of the emotion engine's evaluation and adjusts the content of questions and pace of the interview depending on the candidate's emotional state. For example, if it determines that the candidate is nervous, it selects questions that will help them relax, such as "Tell us about your specific project experience."

[1161] Step 9:

[1162] The terminal presents the next question received from the server to the candidate.

[1163] Step 10:

[1164] This process is repeated until all questions have been completed.

[1165] 3. Evolving learning functions

[1166] Step 1:

[1167] After the interview, the user enters feedback through the system.

[1168] Step 2:

[1169] The terminal collects feedback from the user and transmits it to the server.

[1170] Step 3:

[1171] The server analyzes the collected feedback and interview performance data. Based on the analysis results, the AI ​​model automatically adjusts the questioning style and evaluation criteria. For example, if the feedback states that "there were not enough questions to evaluate logical thinking," the AI ​​model will add more detailed questions to the next interview.

[1172] 4. Interview data analysis

[1173] Step 1:

[1174] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, emotion data from the emotion engine, and feedback.

[1175] Step 2:

[1176] The server analyzes the collected data and provides the user with insights into the interview process, such as analyzing the average technical skill scores of multiple candidates or trends in deficiencies in specific skills, and presents the results to the user in a report.

[1177] Step 3:

[1178] The terminal displays the analysis results provided by the server to the user.

[1179] 5. Providing flexible and scalable solutions

[1180] Step 1:

[1181] The server dynamically adjusts system resources according to the company's hiring volume. For example, if a large number of interviews are being conducted at once, the server automatically increases computing resources.

[1182] Step 2:

[1183] The terminals perform load balancing to ensure stable operation of the system.

[1184] Step 3:

[1185] Users can customize settings for different job types and departments from their terminals.

[1186] Step 4:

[1187] The server generates an appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[1188] This allows companies to use virtual interviewers with an embedded emotion engine to efficiently and effectively recruit top talent.

[1189] Example 2

[1190] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1191] Conventional interview systems have difficulty efficiently evaluating a large number of candidates at once, and customization for specific industries or job types is difficult. Furthermore, they lack the ability to adjust questions in real time to take into account the candidate's emotional state, resulting in inconsistent interview quality. Furthermore, they lack the learning functionality to appropriately reflect feedback and evolve questioning styles and evaluation criteria.

[1192] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for generating a customized virtual interviewer based on user requests; means for conducting real-time dialogue with candidates and conducting Q&A sessions; means for collecting and analyzing interview data to provide an optimized interview experience; means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results; means for evaluating the candidate's emotional state using an emotion analysis engine and adjusting the content of questions in real time; and means for the server to dynamically adjust resources and perform load balancing when multiple interviews are conducted simultaneously. This simplifies customization for specific industries and job types and enables real-time question adjustment taking into account the candidate's emotional state. Furthermore, the quality of interviews can be improved through efficient analysis of interview data and reflection of feedback.

[1193] A "user" is an entity that interacts with the system to customize a virtual interviewer and coordinate and oversee interviews with candidates.

[1194] "Terminal" means the device used by a user or candidate to access the system, including a PC, smartphone, tablet, etc.

[1195] The "server" is a central processing unit that manages and operates the entire system, generates virtual interviewers, answers questions in real time, analyzes data, and evaluates emotional states.

[1196] A "virtual interviewer" is a virtual presence that is customized based on user requirements using generative AI models to interact with candidates and ask questions in real time.

[1197] An "emotion analysis engine" is a software component that analyzes a candidate's facial expressions, tone of voice, choice of words, etc. to assess their emotional state.

[1198] A "generative AI model" is an artificial intelligence algorithm that generates a profile of a virtual interviewer based on user input data and creates question patterns and evaluation criteria.

[1199] A "customization request" is a requirement or condition provided by a user to generate a virtual interviewer suited to a particular industry or job type.

[1200] "Real-time interaction" refers to the process in which a virtual interviewer asks and responds to candidates in real time.

[1201] "Load balancing" is the process by which a server dynamically adjusts computing resources and distributes them to maintain system stability when multiple interviews are running simultaneously.

[1202] "Feedback" refers to opinions and evaluations provided by users through the system after the interview, and is used to optimize questioning style and evaluation criteria.

[1203] "Questioning style" refers to the format and method of asking questions used by the virtual interviewer.

[1204] "Rating criteria" are the standards or metrics that the virtual interviewer uses to evaluate the candidate's skills and attributes.

[1205] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion analysis engine. This system has various functions, such as generating a customized virtual interviewer based on the user's request, conducting real-time Q&A, analyzing and providing interview data, and optimizing question styles based on feedback. Furthermore, by incorporating an emotion analysis engine, it evaluates the candidate's emotional state, providing a more human-like interaction and comprehensive evaluation.

[1206] First, the user inputs a customization request to generate an interviewer for a specific industry and job type through the device's operation screen. This operation screen is provided on devices such as PCs, tablets, and smartphones. For example, the user might input a request such as, "If I'm looking to hire a software engineer, I'd like to evaluate Python and Java programming skills as the skills I want to assess, and problem-solving ability and communication ability as the attributes." The device receives this request and sends it to the server.

[1207] The server uses a generative AI model to generate a virtual interviewer profile based on the user's customization request. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[1208] As a specific prompt, the server sends the following request to the generative AI model: "Please generate a virtual interviewer to conduct job interviews for software engineers. The skills I want to evaluate are programming skills in Python and Java. I also want to evaluate problem-solving ability and communication ability as traits."

[1209] After preparing for the interview, the user uses the virtual interviewer's profile to send an interview invitation to the candidate. The device prepares the necessary equipment so that the candidate can log in at the specified date and time. When the interview starts, the server activates the virtual interviewer and the sentiment analysis engine as well. This allows for real-time question and answer sessions.

[1210] The server asks the candidate an initial question via a virtual interviewer, and the candidate enters their response via their device. The device then sends the response data to the server, where an emotion analysis engine analyzes the candidate's facial expressions, tone of voice, and word choice in real time to assess their emotional state. For example, if a candidate answers, "I have experience with Python and Java," the server adjusts the next question based on the results of the emotion analysis engine. If the candidate is nervous, it can ask questions that will help them relax.

[1211] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the AI ​​model will be informed to add more detailed questions in the next interview.

[1212] Additionally, all data collected during the interview session is stored and analyzed by the server, including the candidate's responses, evaluation scores, collected sentiment data, feedback, etc. The server uses this data to provide insights to the user and provides reports analyzing trends in specific skill deficiencies and average technical skill scores across multiple candidates.

[1213] The server also dynamically adjusts resources and distributes the load to ensure stable system operation even when multiple interviews are conducted at once. This allows terminals to operate stably and multiple interviews to proceed smoothly. This allows companies to efficiently and effectively recruit top talent using virtual interviewers equipped with an emotion analysis engine.

[1214] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1215] Step 1:

[1216] The user inputs a customization request to generate a virtual interviewer for a specific industry and job type through the device's operation screen. For example, for the purpose of hiring a software engineer, a specific request could be input, such as "The skills I want to evaluate are Python and Java programming skills, and the characteristics I want to evaluate are problem-solving ability and communication ability." The input data is sent from the device to the server as a customization request. Input: Industry, Job Type, Evaluation Skills, Evaluation Characteristics | Output: Customization Request

[1217] Step 2:

[1218] The terminal formats the customization request entered by the user and sends it to the server via the network. A specific prompt would be, "Generate a virtual interviewer to conduct a software engineer job interview. The skills we want to evaluate are programming skills in Python and Java. We also want to evaluate problem-solving ability and communication ability as traits." Input: Customization request | Output: Prompt

[1219] Step 3:

[1220] The server receives the customization request and uses the generative AI model to generate a virtual interviewer profile. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is stored in a database. Input: Prompt sentence | Output: Virtual interviewer profile

[1221] Step 4:

[1222] The user uses the generated virtual interviewer profile to send interview invitations to candidates and arrange interview dates. The terminal prepares the candidate for login and prepares the necessary interface. Input: Virtual interviewer profile | Output: Interview invitation

[1223] Step 5:

[1224] The server activates the virtual interviewer at the start time of the interview. The emotion engine is also activated and ready. The server generates the initial questions and asks them to the candidate. Input: Interview invitation | Output: Initial questions

[1225] Step 6:

[1226] Candidates input their answers through their devices. The devices then send the answers to the server, which then uses an emotion engine to analyze the answers. For example, if a candidate answers "I have experience with Python and Java," the server evaluates whether they are speaking with confidence based on their tone of voice and facial expressions. Input: Candidate's answers | Output: Analysis data

[1227] Step 7:

[1228] The server generates the next question based on the results of the sentiment analysis engine. Depending on the candidate's level of nervousness, the server adjusts the questions to present a more relaxed response. The question and answer process is repeated until all questions have been completed. Input: Analysis data | Output: Next question

[1229] Step 8:

[1230] After the interview, the user writes down their feedback, which is then sent to the server. The server analyzes the feedback and optimizes the question style and evaluation criteria for the next interview. Input: Feedback | Output: Optimized question style

[1231] Step 9:

[1232] The server collects and stores data from the entire interview session, performs analysis, and provides insights to the user, including average technical skill scores and trends in specific skill deficiencies. Input: Interview data, analysis data | Output: Report

[1233] Step 10:

[1234] The server dynamically adjusts resources according to the company's hiring volume. When multiple interviews are conducted simultaneously, the server increases computing resources and distributes the load. This ensures stable system operation. Input: Interview schedule | Output: Dynamic resource adjustment

[1235] (Application example 2)

[1236] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1237] Conventional virtual interviewer systems have difficulty taking into account emotional changes during interviews with candidates, resulting in a uniform interview experience. Furthermore, the interviewer's Q&A and evaluations are fixed, resulting in insufficient optimization of the interview. Furthermore, there is insufficient automation of interview data analysis and feedback, which can lead to delays in the evolution of interviewer training and evaluation criteria.

[1238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1239] In this invention, the server includes means for generating a customized virtual interviewer based on user requests, means for conducting real-time dialogue with candidates and conducting Q&A sessions, means for collecting and analyzing interview data to provide an optimized interview experience, means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results, means for analyzing emotional evaluations, and means for adjusting questions based on the analysis results. This enables interactive interviews that take into account the candidate's emotional state, resulting in a high-quality interview experience and efficient talent evaluation.

[1240] "User" refers to an individual or organization that uses this system.

[1241] A "virtual interviewer" refers to a program that uses generative AI models and emotion engines to mimic the role of a real-life interviewer.

[1242] "Real-time interaction" refers to simultaneous interaction between the interviewer and candidate.

[1243] "Question and answer session" refers to the process in which the interviewer asks questions and the candidate answers them.

[1244] "Interview data" refers to information obtained during an interview, such as the candidate's responses, facial expressions, and tone of voice.

[1245] "Analysis" refers to the process of evaluating the collected interview data and deriving meaningful results.

[1246] An "optimized interview experience" refers to an interview process that is tailored to the candidate's emotional state and reactions.

[1247] "Feedback" refers to the evaluation and opinions provided by the user after the interview is completed.

[1248] "Questioning style" refers to the format or pattern of questions used by the interviewer.

[1249] "Evaluation criteria" refers to the standards or criteria for evaluating a candidate's responses or performance.

[1250] "Emotional assessment" refers to the process of analyzing a candidate's facial expressions and tone of voice to determine their emotional state.

[1251] "Analysis results" refer to the specific conclusions or findings obtained after conducting an emotion assessment.

[1252] "Question tailoring" refers to the process of changing or selecting the next question based on the candidate's emotional state.

[1253] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion engine. This system generates a customized virtual interviewer based on the user's request and has functions such as real-time Q&A, analysis and provision of interview data, and optimization of questioning style based on feedback. In addition, by incorporating an emotion engine, it evaluates the candidate's emotional state and provides human-like interaction and comprehensive evaluation.

[1254] System Configuration

[1255] 1. User:

[1256] The user inputs a customization request for generating an interviewer corresponding to a specific industry and job type through the operation screen of the terminal.

[1257] 2. Terminal:

[1258] The terminal collects customization requests from users and sends them to a server. Wearable devices such as smart glasses are used for training and interviewing new operators.

[1259] 3. Server:

[1260] The server uses a generative AI model based on the user's input data to generate a virtual interviewer profile, which includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[1261] Real-time Q&A

[1262] The server activates the virtual interviewer at the start of the interview, and also activates the emotion engine. The server asks the first question through the virtual interviewer, and the candidate inputs their response through their terminal. The terminal sends this response to the server, and the emotion engine analyzes the candidate's facial expressions, tone of voice, choice of words, etc. to evaluate their emotional state.

[1263] For example, if a candidate answers, "I have experience operating machines," the emotion engine evaluates whether they are speaking with confidence based on their tone of voice and facial expressions. The server takes this analysis into account and generates the next question in real time. For example, if the emotion engine determines that the candidate is nervous, the server will present a question that will help them relax, such as, "Tell us about your specific project experience."

[1264] Learning Function

[1265] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the server will reflect this in the AI ​​model and ask more detailed questions in the next interview.

[1266] Data analysis and provision

[1267] All data from the interview session is collected and stored by the server, including the candidate's responses, evaluation scores, collected sentiment data, and feedback. The server analyzes this data and provides insights to the user. For example, it analyzes the average technical skill scores of multiple candidates and trends in specific skill deficiencies, and provides this to the user as a report.

[1268] Providing flexible and scalable solutions

[1269] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the server can automatically increase computing resources and distribute the load to ensure stable operation of the terminals.

[1270] Examples and prompts

[1271] For example, if a new factory operator answers, "I would like to learn how to operate a new machine," during training for new employees, the virtual interviewer will analyze their tone of voice and facial expression, and if it determines that they are nervous, it will put them at ease by asking them the next question, "Please explain the specific operating procedures."

[1272] Example prompt sentence:

[1273] You are a virtual training assistant helping train new employees in a factory. Based on the responses provided by the candidate, you use an emotion engine to analyze their emotional state and tailor the next questions you ask.

[1274] This enables interactive interviews that take into account the candidate's emotional state, resulting in a high-quality interview experience and efficient talent evaluation.

[1275] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1276] Step 1:

[1277] The user operates the device to input customization requests to generate a virtual interviewer for a specific industry and job type. The data input by the user includes the skills (e.g., programming skills, knowledge of machine operations) and characteristics (e.g., problem-solving ability, communication ability) to be evaluated. The input data is sent to the server in JSON format.

[1278] Step 2:

[1279] The device collects customization requests from the user and sends them to the server. The data sent as the customization request includes the user's input information. Upon receiving this data, the server uses a generative AI model to generate a profile for the virtual interviewer. The profile includes question patterns and evaluation criteria appropriate for the industry and job type.

[1280] Step 3:

[1281] The server saves the generated virtual interviewer profile and prepares for the interview. The profile saves specific questions and evaluation criteria in template format. The saved profile is used in subsequent processing.

[1282] Step 4:

[1283] After the interview date is arranged, the terminal sends an interview invitation to the candidate, and prepares the candidate so that they can log in at the specified date and time. The server verifies the login information and activates the virtual interviewer at the start time of the interview.

[1284] Step 5:

[1285] When the interview begins, the server asks the first question via the virtual interviewer, and the candidate enters their response via the terminal. The entered response data is sent in text format to the server, which then receives the candidate's response and analyzes the data using a natural language processing engine.

[1286] Step 6:

[1287] The server receives the candidate's responses and uses an emotion engine to analyze the candidate's facial expressions, tone of voice, and word choice. Based on the analysis results, the server evaluates the candidate's emotional state (e.g., nervous, confident, confused). The evaluation data is used to generate the next questions in real time.

[1288] Step 7:

[1289] Taking into account the analysis results of the emotion engine, the server generates the next question in real time. For example, if the server determines that the candidate is nervous, it will present a question that will help them relax, such as "Tell us about your specific project experience." The generated question is displayed to the candidate via their device.

[1290] Step 8:

[1291] The question and answer process is repeated until all questions are completed. The server records all collected data and stores all data from the interview session, including the candidate's responses, evaluation scores, collected emotional data, and feedback.

[1292] Step 9:

[1293] After the interview, the user enters feedback through the system. The terminal collects the feedback and sends it to the server. The server analyzes the collected feedback and optimizes the question style and evaluation criteria for the next interview, thereby improving the accuracy and effectiveness of the virtual interviewer.

[1294] Step 10:

[1295] The server generates an interview report based on the analysis results and provides it to the user. The report includes the average technical skill scores of multiple candidates, trends in specific skill deficiencies, and an overall evaluation of the interview. Users can use this report to optimize their recruitment activities.

[1296] The above are the specific processing steps for implementing the present invention, which enable interactive interviews that take into account the candidate's emotional state, thereby achieving a high-quality interview experience and efficient talent evaluation.

[1297] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1298] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1299] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1300] [Fourth embodiment]

[1301] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1302] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1303] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1304] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1305] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1306] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1307] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1308] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1309] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1310] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1311] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1312] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1313] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1314] The present invention will be described below as a specific method for implementing the invention, with reference to the operation of a system using a user, a terminal, and a server. The invention is implemented based on the following configuration.

[1315] 1. Interviewer generation and customization

[1316] 1-1: Receive customization requests

[1317] Users use the system to submit customization requests to generate interviewers for specific industries and job types. For example, if the user is looking to hire a software engineer, the user inputs the skills (e.g., programming skills and algorithm knowledge) and characteristics (e.g., problem-solving ability and communication ability) they want to evaluate on the device's operation screen. The device then collects this information and sends it to the server as a customization request.

[1318] 1-2: Generate interviewers

[1319] The server analyzes the customization request sent by the user and generates a virtual interviewer using a generative AI model based on the analysis. The virtual interviewer has question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[1320] 2. Real-time Q&A

[1321] 2-1: Start of the interview

[1322] The user sends an interview invitation to the candidate and schedules the interview. The terminal is prepared so that the candidate can log in to the system at the specified date and time. The server activates the virtual interviewer at the start time of the interview.

[1323] 2-2: Providing questions and recording responses

[1324] The server asks the candidate an initial question via a virtual interviewer. For example, "Tell me about your programming experience." When the candidate enters their response via their terminal, it is sent to the server. The server analyzes the candidate's response and generates the next question in real time based on the content. For example, if the candidate answers "I have experience with Python and Java," the server generates the next question in real time: "In what specific projects have you used those languages?" This allows for a detailed evaluation of the candidate's skills and aptitude.

[1325] 3. Evolving learning functions

[1326] 3-1: Collecting and sending feedback

[1327] After the interview, the user enters feedback through the system, including an evaluation of the overall interview and evaluation of specific questions and responses. The terminal collects this feedback and sends it to the server.

[1328] 3-2: Optimizing question style and evaluation criteria

[1329] The server analyzes the collected feedback and interview performance data. For example, if feedback indicates that there are not enough questions to evaluate the candidate's logical thinking, the system can use this information to adjust the questioning style and add a question such as "Please explain the specific steps you took to solve the problem" to the next interview. In this way, the virtual interviewer can continuously evolve and provide more effective interviews.

[1330] 4. Interview data analysis

[1331] 4-1: Collection of interview data

[1332] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback.

[1333] 4-2: Analyzing data and providing insights

[1334] The server analyzes the collected data and provides the user with insights into the interview process, such as analyzing the average technical skill scores of multiple candidates or specific skill deficiencies, and generates a report that is provided to the user, allowing the user to gain specific insights to further optimize the hiring process.

[1335] 5. Providing flexible and scalable solutions

[1336] 5-1: Scaling up

[1337] The server dynamically adjusts system resources according to the company's hiring volume. For example, if a large number of interviews are being conducted at once, the necessary computing resources will be automatically increased. The server also distributes the load to ensure stable operation of terminals.

[1338] 5-2: Meeting different hiring needs

[1339] Users can use their devices to customize settings for different job types and departments. For example, they can set up the system to generate virtual interviewers that are optimal for each job type, so that it can accommodate not only software engineers but also marketing and sales positions. The server generates the appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[1340] The above is an embodiment of the present invention, which enables companies to recruit talented personnel efficiently and effectively.

[1341] The processing flow will be explained below.

[1342] 1. Interviewer generation and customization

[1343] Step 1:

[1344] The user inputs the industry, job type, and skills and characteristics they want to evaluate through the device's operation screen.

[1345] Step 2:

[1346] The terminal collects input information from the user and transmits it to the server as a customization request.

[1347] Step 3:

[1348] The server analyzes the received request and uses a generative AI model to generate a profile for the virtual interviewer, which includes question patterns and evaluation criteria appropriate for the industry and job type.

[1349] Step 4:

[1350] The server stores the generated virtual interviewer profile and the interview is ready.

[1351] 2. Real-time Q&A

[1352] Step 1:

[1353] The user sends interview invitations to candidates and schedules interviews.

[1354] Step 2:

[1355] The terminal will be ready for the candidate to log into the system at the appointed date and time.

[1356] Step 3:

[1357] The server activates the virtual interviewer at the start time of the interview.

[1358] Step 4:

[1359] The server asks the candidate an initial question via a virtual interviewer.

[1360] Step 5:

[1361] The candidate enters their response through a terminal.

[1362] Step 6:

[1363] The terminal sends the candidate's response to the server.

[1364] Step 7:

[1365] The server analyzes the candidate's responses and generates subsequent questions in real time based on their content. For example, if a question about technical skills follows, the server generates a question such as, "What specific projects have you used those languages ​​in?"

[1366] Step 8:

[1367] The server generates the next question and sends it to the terminal, which then presents it to the candidate.

[1368] Step 9:

[1369] This process is repeated until all questions have been completed.

[1370] 3. Evolving learning functions

[1371] Step 1:

[1372] After the interview, the user enters feedback through the system.

[1373] Step 2:

[1374] The device collects the feedback and sends it to the server.

[1375] Step 3:

[1376] The server analyzes the collected feedback and interview performance data. Based on the analysis results, the question style and evaluation criteria are automatically adjusted according to the AI ​​model. For example, if feedback indicates that there are a lack of questions that evaluate logical thinking, relevant question patterns are added.

[1377] 4. Interview data analysis

[1378] Step 1:

[1379] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback.

[1380] Step 2:

[1381] The server analyzes the collected data and provides the user with insight into the interview process, such as average technical skill scores or identifying skill gaps, and generates a report of the results.

[1382] Step 3:

[1383] The terminal displays the analysis results provided by the server to the user.

[1384] 5. Providing flexible and scalable solutions

[1385] Step 1:

[1386] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the computing resources will be automatically increased.

[1387] Step 2:

[1388] Terminals distribute the load to ensure stable operation of the system.

[1389] Step 3:

[1390] Users can customize settings for different job types or departments from their terminals.

[1391] Step 4:

[1392] The server generates an appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[1393] The above are the specific processing steps of the present invention, and this flow enables companies to recruit talented personnel efficiently and effectively.

[1394] Example 1

[1395] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1396] In the modern recruitment process, there are many challenges in candidate evaluation and interview efficiency. In particular, evaluations are often influenced by the interviewer's subjectivity, preparing for and conducting interviews requires a great deal of time and resources, and continuous improvement based on feedback is difficult. There is a need to solve these challenges and provide a more effective and efficient interview process.

[1397] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1398] In this invention, the server includes a means for generating a customized virtual interviewer based on user requests, a means for conducting real-time dialogue with candidates and conducting Q&A sessions, and a means for collecting and analyzing interview data to provide an optimized interview experience. This streamlines the interview process and enables fair and objective candidate evaluation. Furthermore, the virtual interviewer's questioning style and evaluation criteria can be automatically learned using user feedback and interview results, enabling continuous improvement. Furthermore, because the customized virtual interviewer is generated using a generative AI model, it is easy to create a virtual interviewer with question patterns and evaluation criteria suited to specific industries and job types.

[1399] A "user request" is a request by a user to customize the virtual interviewer based on a particular industry, job type, and assessment skills when generating the virtual interviewer.

[1400] A "customized virtual interviewer" is a virtual interviewer with question patterns and evaluation criteria suited to the needs of a specific industry or job type, according to the user's request.

[1401] A "generative AI model" is an artificial intelligence model that analyzes user requirements and generates a customized virtual interviewer based on those requirements.

[1402] "Real-time dialogue" refers to the process of an interview in which the candidate and virtual interviewer interactively exchange questions and answers, with immediate responses and feedback.

[1403] A "question and answer" is a series of interactions in which a virtual interviewer asks a candidate questions and the candidate responds.

[1404] "Interview data" refers to data collected during an interview, including information such as the candidate's responses, evaluation scores, and feedback.

[1405] An "optimized interview experience" refers to an interview process that improves the quality and efficiency of interviews by analyzing interview data and utilizing feedback.

[1406] "Feedback" refers to the evaluation of the entire interview provided by the user after the interview is completed, as well as evaluation comments on specific questions and answers.

[1407] "Questioning style" is a concept that refers to the format and manner in which a virtual interviewer asks questions to a candidate.

[1408] "Evaluation criteria" are indicators or standards that the virtual interviewer uses to evaluate the candidate's responses, and are set based on the user's requests.

[1409] A "profile" refers to a generated set of information about a virtual interviewer, including data related to interview preparation.

[1410] "Analysis" is the process of analyzing the collected interview data and generating insights and recommendations based on the results.

[1411] The present invention is an interview support system implemented using a user, a terminal, and a server, and a specific implementation method thereof will be described below. Implementing the invention requires cooperation between hardware and software.

[1412] 1. Interviewer generation and customization

[1413] To generate a virtual interviewer for a specific industry and job type, the user inputs customization requests using the device's operation screen. Examples of input include evaluation items such as "programming skills," "algorithm knowledge," "problem-solving ability," and "communication ability." The device collects this input information and sends it to the server as a customization request.

[1414] The server analyzes the received customization request and uses a generative AI model to generate a customized virtual interviewer with question patterns and evaluation criteria suited to the specific industry and job type. The generated virtual interviewer profile is stored on the server and is ready for the interview.

[1415] 2. Real-time Q&A

[1416] The user sends an interview invitation to the candidate from the terminal and arranges the interview date. The terminal makes the necessary preparations so that the candidate can log in to the system at the specified date and time. The server activates the virtual interviewer at the start time of the interview.

[1417] When the interview begins, the server provides the candidate with the first question through the virtual interviewer. A specific example question is, "Tell me about your programming experience." The candidate enters their response through their terminal, which is then sent to the server. The server analyzes the candidate's response and generates the next question in real time based on the content. For example, if the candidate answers, "I have experience with Python and Java," the server generates a question such as, "In what specific projects have you used those languages?"

[1418] 3. Evolving learning functions

[1419] After the interview, the user enters feedback through the system. The feedback includes an evaluation of the entire interview and evaluation of specific questions and answers. The terminal collects this feedback and sends it to the server.

[1420] The server analyzes the collected feedback and interview performance data to optimize the virtual interviewer's questioning style and evaluation criteria. For example, if feedback indicates that there are not enough questions to evaluate the candidate's logical thinking, the server will add a question such as "Please explain the specific steps you took to solve the problem" to the next interview. In this way, the virtual interviewer continuously evolves and provides more effective interviews.

[1421] 4. Interview data analysis

[1422] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, and feedback. The server analyzes the collected data and provides users with insights into the interview process. For example, the average technical skill scores of multiple candidates and specific skill deficiencies are analyzed and provided to users as a report. This allows users to gain specific insights to further optimize the hiring process.

[1423] 5. Providing flexible and scalable solutions

[1424] The server dynamically adjusts system resources according to the company's hiring volume. For example, when conducting a large number of interviews at once, it automatically increases the necessary computing resources and distributes the load to ensure stable terminal operation. Users can also use their terminals to customize settings for different job types and departments. For example, it is possible to set up the system to generate virtual interviewers optimized for each job type, so that it can accommodate not only software engineers but also marketing and sales positions. The server generates appropriate virtual interviewers based on each request and supports the smooth conduct of interviews.

[1425] Specific examples

[1426] An example prompt might be, "What skills are necessary to generate software engineer interviewers?", to which the system generates specific answers such as "programming skills, knowledge of algorithms, problem-solving ability, and communication skills."

[1427] By following the above procedure, the present invention can be implemented, enabling companies to recruit talented personnel efficiently and effectively.

[1428] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1429] Step 1: Enter your customization requests

[1430] The user inputs a request for customizing the virtual interviewer using the operation screen of the terminal.

[1431] As an example of input, select evaluation items such as "programming skills" and "knowledge of algorithms."

[1432] The terminal collects the input customization requests and transmits them to the server as a customization request.

[1433] Input: The customization request that the user types into the terminal.

[1434] Output: Customization request data generated by the device

[1435] Step 2: Parsing the customization request

[1436] The server receives a customization request sent from the terminal.

[1437] The server analyzes the received data and extracts parameters to input into the generative AI model.

[1438] Data analysis involves classifying and organizing information based on industry, job type, and assessed skills.

[1439] Input: Customization request sent from the device

[1440] Output: Analysis results to be input into the generative AI model

[1441] Step 3: Generate a virtual interviewer

[1442] The server generates a virtual interviewer using a generative AI model based on the analysis results.

[1443] The generated virtual interviewer has a specific question pattern and evaluation criteria.

[1444] The generated virtual interviewer profile is stored in the server.

[1445] Input: Analysis results (parameters reflecting user customization requests)

[1446] Output: Virtual interviewer profile

[1447] Step 4: Schedule and prepare for the interview

[1448] The user operates the terminal to set an interview date in order to send an interview invitation to the candidate.

[1449] The terminal transmits the set schedule information to the server and prepares for the candidate to log in at the specified date and time.

[1450] The server activates the virtual interviewer at the start time of the interview.

[1451] Input: Interview schedule information

[1452] Output: Sending interview invitations and preparing the system

[1453] Step 5: Start the interview and ask questions

[1454] The server provides the candidate with initial questions through a virtual interviewer.

[1455] The server generates questions based on the evaluation items set by the user.

[1456] The candidate enters their response through the terminal, which is then sent to the server.

[1457] Input: Candidate login and responses to questions

[1458] Output: Candidate response data

[1459] Step 6: Parse the response and generate the next question

[1460] The server analyzes the candidate's responses and generates the next question based on the content in real time.

[1461] For example, if you answer "I have experience with Python and Java," questions like "What specific projects have you used those languages ​​in?" will be generated.

[1462] Input: Candidate response data

[1463] Output: Next question

[1464] Step 7: Gather feedback

[1465] After the interview, the user uses the terminal to enter feedback.

[1466] Feedback may include an evaluation of the interview as a whole and an evaluation of specific questions and responses.

[1467] The terminal sends the feedback to the server.

[1468] Input: User feedback information

[1469] Output: Feedback data

[1470] Step 8: Feedback analysis and optimization

[1471] The server analyzes the collected feedback and interview performance data.

[1472] The server optimizes the virtual interviewer's questioning style and evaluation criteria based on the analysis results.

[1473] For example, if you receive feedback that there are not enough questions to assess logical thinking, add a question such as "Please explain the specific steps you took to solve the problem" to your next interview.

[1474] Input: Feedback data and interview performance data

[1475] Output: Optimized question style and evaluation criteria

[1476] Step 9: Store and analyze the interview data

[1477] The server collects and stores data from all interview sessions.

[1478] The server analyses the collected data and provides the user with insight into the interview process.

[1479] Input: Interview session data

[1480] Output: Interview process insights and reports

[1481] Step 10: Dynamically Adjust System Resources

[1482] The server dynamically adjusts system resources according to the company's adoption volume.

[1483] When multiple interviews are conducted at once, the server automatically increases computing resources and distributes the load to ensure stable terminal operation.

[1484] Input: Change in recruitment volume

[1485] Output: Dynamically adjusted system resources

[1486] (Application example 1)

[1487] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1488] In modern factories, accurately evaluating the capabilities of automated robots and efficiently training them are key challenges. Robots with diverse functions in particular need to have their skills evaluated and improved appropriately, but existing systems make it difficult to customize individual evaluation criteria and training scenarios, making it difficult to ensure an optimal training process.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1490] In this invention, the server includes a means for generating a customized virtual trainer based on a user's request, a means for conducting real-time interaction with the trainee device and performing skill evaluation, and a means for collecting and analyzing training data to provide an optimized training experience, thereby enabling accurate evaluation of the robot's capabilities and providing adaptive and efficient training.

[1491] A "user" is an individual or group that operates the system or requests customization.

[1492] A "virtual trainer" is a virtual role model that is generated based on a user's request and performs skill evaluation and training on the trainee device.

[1493] "Training equipment" refers to machinery and equipment that is the subject of training and evaluation, such as robots and automated operating devices used in factories.

[1494] "Real-time interaction" refers to the process in which the virtual trainer interacts directly with the trainee device in real time, exchanging questions and instructions in real time.

[1495] "Skills evaluation" is the act of measuring and evaluating abilities such as the skill to operate the training equipment, reaction speed, and error handling ability.

[1496] "Training Data" means data collected during a training session, including the response, performance, and feedback of the trained device.

[1497] An "optimized training experience" is a training session that is customized to the characteristics of the user and the equipment being trained, and is delivered efficiently and effectively.

[1498] "Feedback" refers to ratings and comments provided by users and those involved in the training, which are used to improve the virtual trainer and optimize the training.

[1499] The "question style" refers to the format or method of questions that the virtual trainer asks the trainee device.

[1500] "Evaluation criteria" are the standards or metrics used to evaluate the trainee's skills.

[1501] The present invention provides a system for effectively training and evaluating robots in factories. An embodiment of the system will be described in detail below.

[1502] The system primarily consists of a user, a terminal, and a server. The user, such as a factory staff member, inputs the characteristics and evaluation criteria of the robot they want to train. The terminal is a computer, tablet, or smartphone operated by the user, and is responsible for collecting information from these and sending it to the server. The server uses a generative AI model to generate a virtual trainer and evaluates the robot's operation and performs training in real time.

[1503] Program processing

[1504] The server performs the process as follows:

[1505] 1. Receive user customization requests

[1506] The user inputs the skills they want to evaluate (e.g., operation skills, reaction speed, error handling ability) through the terminal, and this information is sent to the server.

[1507] 2. Creating a Virtual Trainer

[1508] The server uses the OpenAI API to generate prompts based on the user's requests. For example, the customization request is converted into a prompt, such as "User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, error handling ability." The server then uses the generative AI model to generate a virtual trainer.

[1509] 3. Training commencement and evaluation

[1510] The server activates a virtual trainer, which interacts with the trainee in real time. The virtual trainer presents the robot with a series of questions and tasks, and collects and evaluates its responses. For example, it runs scenarios to evaluate skills such as object recognition, obstacle avoidance, and efficient route planning.

[1511] 4. Data Collection and Optimization

[1512] Data collected during the training session (robot responses, performance data, user feedback, etc.) is stored and analyzed on a server. Based on the results of this analysis, the virtual trainer's questioning style and evaluation criteria are optimized.

[1513] Hardware and software used

[1514] Hardware: Factory robots (e.g., transport robots, assembly robots), user-operated devices (computers, tablets, smartphones)

[1515] Software: OpenAI API, Python program

[1516] Specific examples

[1517] When training a transport robot in a factory, the user inputs evaluation skills using a terminal. For example, if they want to evaluate "operation skills, reaction speed, and error handling ability," this is converted into prompt statements and sent to the server. The virtual trainer presents the robot with a scenario such as, "To evaluate your operation skills, please recognize and transport objects accurately." The robot's behavior is evaluated in real time, and after training is complete, it is optimized based on the feedback.

[1518] Example prompt sentence:

[1519] User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, error handling ability

[1520] Scenario: The virtual trainer sets up an evaluation scenario for the transport robot and records its responses in real time.

[1521] This embodiment allows accurate evaluation of the capabilities of robots in a factory and effective training.

[1522] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1523] Step 1:

[1524] The user inputs the skills they want to evaluate and the characteristics of the robot they are training through their device. For example, the user specifies "modelXYZ" as the robot they want to evaluate for its "operation skills, reaction speed, and error handling ability." This input information is sent from the device to the server.

[1525] Step 2:

[1526] The server generates a prompt based on the received user customization request. The generated prompt is "User ID: user123, Robot model: modelXYZ, Evaluation skills: Operation skills, reaction speed, and error handling ability." This provides the specific data required to generate a virtual trainer.

[1527] Step 3:

[1528] The server generates a virtual trainer using the generative AI model. Specifically, it calls the OpenAI API and provides prompts as input data. The API response generates scripts and scenarios for the virtual trainer, which are then stored on the server.

[1529] Step 4:

[1530] The server activates the virtual trainer to start training. The server sends a training start command to the terminal, and the terminal sends a training start signal to the trained device (robot). At this point, the virtual trainer begins to develop a scenario suitable for the model XYZ robot.

[1531] Step 5:

[1532] A virtual trainer sets up evaluation scenarios for the robot in real time, presenting questions and tasks to the robot, which then perform specific tasks such as object recognition, obstacle avoidance, and efficient route planning. This process generates data on the robot's responses.

[1533] Step 6:

[1534] The server collects the robot's response data generated in real time, including task completion time, number of errors, and performance indicators. This data is stored on the server as training results.

[1535] Step 7:

[1536] The user inputs feedback on the training results through the device. For example, they can provide specific feedback such as, "The robot's reaction speed is appropriate, but its obstacle avoidance is insufficient." This feedback is then sent from the device to the server.

[1537] Step 8:

[1538] The server analyzes user feedback and collected training data to optimize the virtual trainer's questioning style and evaluation criteria, thereby providing a more adaptive and efficient scenario for the next training session.

[1539] Step 9:

[1540] The server updates the optimized virtual trainer profile and makes it available for the next use, thereby achieving continuous training and assessment improvement.

[1541] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1542] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion engine. This system has various functions, such as generating a customized virtual interviewer based on the user's request, conducting real-time Q&A, analyzing and providing interview data, and optimizing question styles based on feedback. Furthermore, by incorporating an emotion engine, the system evaluates the candidate's emotional state, providing more human-like interaction and comprehensive evaluation.

[1543] 1. Interviewer generation and customization

[1544] The user inputs a customization request to generate an interviewer for a specific industry and job type through the device's operation screen. For example, if the goal is to hire a software engineer, the user inputs the skills (programming skills and algorithm knowledge) and characteristics (problem-solving ability and communication ability) they want to evaluate. The device collects this information and sends it to the server as a customization request.

[1545] The server uses a generative AI model to generate a virtual interviewer profile based on the user's input data. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[1546] 2. Real-time Q&A

[1547] The user sends an interview invitation to the candidate and schedules the interview. The terminal is prepared so that the candidate can log in at the specified date and time. The server activates the virtual interviewer at the start time of the interview, and also activates the emotion engine.

[1548] The server asks the first question via a virtual interviewer, and the candidate inputs their response via their device. The device then sends this response to the server, where the emotion engine analyzes the candidate's facial expressions, tone of voice, and choice of words to evaluate their emotional state. For example, if a candidate answers, "I have experience with Python and Java," the tone of voice and facial expressions will be used to evaluate whether they are speaking with confidence.

[1549] The server takes into account the analysis results of the emotion engine and generates the next question in real time. The next question is adjusted according to the candidate's emotional state. For example, if the emotion engine determines that the candidate is nervous, the server will present a relaxing question such as "Tell us about your specific project experience."

[1550] This question and answer process is repeated until all questions are answered, and all data collected by the server is recorded.

[1551] 3. Evolving learning functions

[1552] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the server will reflect this in the AI ​​model and ask more detailed questions in the next interview.

[1553] 4. Interview data analysis

[1554] All data from the interview session is collected and stored by the server, including the candidate's responses, evaluation scores, collected sentiment data, and feedback. The server analyzes this data and provides insights to the user, such as average technical skill scores across multiple candidates or trends in specific skill deficiencies, and provides these to the user in the form of a report.

[1555] 5. Providing flexible and scalable solutions

[1556] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the computing resources will be automatically increased. This allows for load balancing to ensure stable operation of terminals.

[1557] Users can customize the settings from their devices for different job types and departments. For example, they can create virtual interviewers for marketing and sales positions, as well as software engineers. The server generates virtual interviewers based on each requirement and helps ensure the interview process goes smoothly.

[1558] The above is an embodiment of the present invention, which enables companies to efficiently and effectively recruit talented personnel by using a virtual interviewer incorporating an emotion engine.

[1559] The processing flow will be explained below.

[1560] 1. Interviewer generation and customization

[1561] Step 1:

[1562] The user inputs a customization request for generating an interviewer corresponding to a specific industry and job type via the operation screen of the terminal.

[1563] Step 2:

[1564] The terminal collects the information input by the user and transmits it to the server as a customization request.

[1565] Step 3:

[1566] The server analyzes the requests received from users and uses a generative AI model to generate a profile for the virtual interviewer, which includes question patterns and evaluation criteria appropriate for the industry and job type.

[1567] Step 4:

[1568] The server stores the generated profile of the virtual interviewer, and the interview preparation is complete.

[1569] 2. Real-time Q&A

[1570] Step 1:

[1571] The user sends interview invitations to candidates and schedules interview dates.

[1572] Step 2:

[1573] The terminal will be ready for the candidate to log into the system at the specified date and time.

[1574] Step 3:

[1575] The server activates the virtual interviewer and the emotion engine in accordance with the start time of the interview.

[1576] Step 4:

[1577] The server asks the first question via a virtual interviewer.

[1578] Step 5:

[1579] The candidate enters the response through a terminal.

[1580] Step 6:

[1581] The terminal transmits the candidate's response to the server.

[1582] Step 7:

[1583] The server analyzes the candidate's responses in real time and generates the next question based on the content, while an emotion engine is initiated to analyze the candidate's facial expressions, tone of voice, choice of words, etc. to evaluate their emotional state.

[1584] Step 8:

[1585] The server considers the results of the emotion engine's evaluation and adjusts the content of questions and pace of the interview depending on the candidate's emotional state. For example, if it determines that the candidate is nervous, it selects questions that will help them relax, such as "Tell us about your specific project experience."

[1586] Step 9:

[1587] The terminal presents the next question received from the server to the candidate.

[1588] Step 10:

[1589] This process is repeated until all questions have been completed.

[1590] 3. Evolving learning functions

[1591] Step 1:

[1592] After the interview, the user enters feedback through the system.

[1593] Step 2:

[1594] The terminal collects feedback from the user and transmits it to the server.

[1595] Step 3:

[1596] The server analyzes the collected feedback and interview performance data. Based on the analysis results, the AI ​​model automatically adjusts the questioning style and evaluation criteria. For example, if the feedback states that "there were not enough questions to evaluate logical thinking," the AI ​​model will add more detailed questions to the next interview.

[1597] 4. Interview data analysis

[1598] Step 1:

[1599] The server collects and stores data from all interview sessions, including candidate responses, assessment scores, emotion data from the emotion engine, and feedback.

[1600] Step 2:

[1601] The server analyzes the collected data and provides the user with insights into the interview process, such as analyzing the average technical skill scores of multiple candidates or trends in deficiencies in specific skills, and presents the results to the user in a report.

[1602] Step 3:

[1603] The terminal displays the analysis results provided by the server to the user.

[1604] 5. Providing flexible and scalable solutions

[1605] Step 1:

[1606] The server dynamically adjusts system resources according to the company's hiring volume. For example, if a large number of interviews are being conducted at once, the server automatically increases computing resources.

[1607] Step 2:

[1608] The terminals perform load balancing to ensure stable operation of the system.

[1609] Step 3:

[1610] Users can customize settings for different job types and departments from their terminals.

[1611] Step 4:

[1612] The server generates an appropriate virtual interviewer based on each request and supports the smooth conduct of the interview.

[1613] This allows companies to use virtual interviewers with an embedded emotion engine to efficiently and effectively recruit top talent.

[1614] Example 2

[1615] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1616] Conventional interview systems have difficulty efficiently evaluating a large number of candidates at once, and customization for specific industries or job types is difficult. Furthermore, they lack the ability to adjust questions in real time to take into account the candidate's emotional state, resulting in inconsistent interview quality. Furthermore, they lack the learning functionality to appropriately reflect feedback and evolve questioning styles and evaluation criteria.

[1617] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for generating a customized virtual interviewer based on user requests; means for conducting real-time dialogue with candidates and conducting Q&A sessions; means for collecting and analyzing interview data to provide an optimized interview experience; means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results; means for evaluating the candidate's emotional state using an emotion analysis engine and adjusting the content of questions in real time; and means for the server to dynamically adjust resources and perform load balancing when multiple interviews are conducted simultaneously. This simplifies customization for specific industries and job types and enables real-time question adjustment taking into account the candidate's emotional state. Furthermore, the quality of interviews can be improved through efficient analysis of interview data and reflection of feedback.

[1618] A "user" is an entity that interacts with the system to customize a virtual interviewer and coordinate and oversee interviews with candidates.

[1619] "Terminal" means the device used by a user or candidate to access the system, including a PC, smartphone, tablet, etc.

[1620] The "server" is a central processing unit that manages and operates the entire system, generates virtual interviewers, answers questions in real time, analyzes data, and evaluates emotional states.

[1621] A "virtual interviewer" is a virtual presence that is customized based on user requirements using generative AI models to interact with candidates and ask questions in real time.

[1622] An "emotion analysis engine" is a software component that analyzes a candidate's facial expressions, tone of voice, choice of words, etc. to assess their emotional state.

[1623] A "generative AI model" is an artificial intelligence algorithm that generates a profile of a virtual interviewer based on user input data and creates question patterns and evaluation criteria.

[1624] A "customization request" is a requirement or condition provided by a user to generate a virtual interviewer suited to a particular industry or job type.

[1625] "Real-time interaction" refers to the process in which a virtual interviewer asks and responds to candidates in real time.

[1626] "Load balancing" is the process by which a server dynamically adjusts computing resources and distributes them to maintain system stability when multiple interviews are running simultaneously.

[1627] "Feedback" refers to opinions and evaluations provided by users through the system after the interview, and is used to optimize questioning style and evaluation criteria.

[1628] "Questioning style" refers to the format and method of asking questions used by the virtual interviewer.

[1629] "Rating criteria" are the standards or metrics that the virtual interviewer uses to evaluate the candidate's skills and attributes.

[1630] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion analysis engine. This system has various functions, such as generating a customized virtual interviewer based on the user's request, conducting real-time Q&A, analyzing and providing interview data, and optimizing question styles based on feedback. Furthermore, by incorporating an emotion analysis engine, it evaluates the candidate's emotional state, providing a more human-like interaction and comprehensive evaluation.

[1631] First, the user inputs a customization request to generate an interviewer for a specific industry and job type through the device's operation screen. This operation screen is provided on devices such as PCs, tablets, and smartphones. For example, the user might input a request such as, "If I'm looking to hire a software engineer, I'd like to evaluate Python and Java programming skills as the skills I want to assess, and problem-solving ability and communication ability as the attributes." The device receives this request and sends it to the server.

[1632] The server uses a generative AI model to generate a virtual interviewer profile based on the user's customization request. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[1633] As a specific prompt, the server sends the following request to the generative AI model: "Please generate a virtual interviewer to conduct job interviews for software engineers. The skills I want to evaluate are programming skills in Python and Java. I also want to evaluate problem-solving ability and communication ability as traits."

[1634] After preparing for the interview, the user uses the virtual interviewer's profile to send an interview invitation to the candidate. The device prepares the necessary equipment so that the candidate can log in at the specified date and time. When the interview starts, the server activates the virtual interviewer and the sentiment analysis engine as well. This allows for real-time question and answer sessions.

[1635] The server asks the candidate an initial question via a virtual interviewer, and the candidate enters their response via their device. The device then sends the response data to the server, where an emotion analysis engine analyzes the candidate's facial expressions, tone of voice, and word choice in real time to assess their emotional state. For example, if a candidate answers, "I have experience with Python and Java," the server adjusts the next question based on the results of the emotion analysis engine. If the candidate is nervous, it can ask questions that will help them relax.

[1636] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the AI ​​model will be informed to add more detailed questions in the next interview.

[1637] Additionally, all data collected during the interview session is stored and analyzed by the server, including the candidate's responses, evaluation scores, collected sentiment data, feedback, etc. The server uses this data to provide insights to the user and provides reports analyzing trends in specific skill deficiencies and average technical skill scores across multiple candidates.

[1638] The server also dynamically adjusts resources and distributes the load to ensure stable system operation even when multiple interviews are conducted at once. This allows terminals to operate stably and multiple interviews to proceed smoothly. This allows companies to efficiently and effectively recruit top talent using virtual interviewers equipped with an emotion analysis engine.

[1639] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1640] Step 1:

[1641] The user inputs a customization request to generate a virtual interviewer for a specific industry and job type through the device's operation screen. For example, for the purpose of hiring a software engineer, a specific request could be input, such as "The skills I want to evaluate are Python and Java programming skills, and the characteristics I want to evaluate are problem-solving ability and communication ability." The input data is sent from the device to the server as a customization request. Input: Industry, Job Type, Evaluation Skills, Evaluation Characteristics | Output: Customization Request

[1642] Step 2:

[1643] The terminal formats the customization request entered by the user and sends it to the server via the network. A specific prompt would be, "Generate a virtual interviewer to conduct a software engineer job interview. The skills we want to evaluate are programming skills in Python and Java. We also want to evaluate problem-solving ability and communication ability as traits." Input: Customization request | Output: Prompt

[1644] Step 3:

[1645] The server receives the customization request and uses the generative AI model to generate a virtual interviewer profile. This profile includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is stored in a database. Input: Prompt sentence | Output: Virtual interviewer profile

[1646] Step 4:

[1647] The user uses the generated virtual interviewer profile to send interview invitations to candidates and arrange interview dates. The terminal prepares the candidate for login and prepares the necessary interface. Input: Virtual interviewer profile | Output: Interview invitation

[1648] Step 5:

[1649] The server activates the virtual interviewer at the start time of the interview. The emotion engine is also activated and ready. The server generates the initial questions and asks them to the candidate. Input: Interview invitation | Output: Initial questions

[1650] Step 6:

[1651] Candidates input their answers through their devices. The devices then send the answers to the server, which then uses an emotion engine to analyze the answers. For example, if a candidate answers "I have experience with Python and Java," the server evaluates whether they are speaking with confidence based on their tone of voice and facial expressions. Input: Candidate's answers | Output: Analysis data

[1652] Step 7:

[1653] The server generates the next question based on the results of the sentiment analysis engine. Depending on the candidate's level of nervousness, the server adjusts the questions to present a more relaxed response. The question and answer process is repeated until all questions have been completed. Input: Analysis data | Output: Next question

[1654] Step 8:

[1655] After the interview, the user writes down their feedback, which is then sent to the server. The server analyzes the feedback and optimizes the question style and evaluation criteria for the next interview. Input: Feedback | Output: Optimized question style

[1656] Step 9:

[1657] The server collects and stores data from the entire interview session, performs analysis, and provides insights to the user, including average technical skill scores and trends in specific skill deficiencies. Input: Interview data, analysis data | Output: Report

[1658] Step 10:

[1659] The server dynamically adjusts resources according to the company's hiring volume. When multiple interviews are conducted simultaneously, the server increases computing resources and distributes the load. This ensures stable system operation. Input: Interview schedule | Output: Dynamic resource adjustment

[1660] (Application example 2)

[1661] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1662] Conventional virtual interviewer systems have difficulty taking into account emotional changes during interviews with candidates, resulting in a uniform interview experience. Furthermore, the interviewer's Q&A and evaluations are fixed, resulting in insufficient optimization of the interview. Furthermore, there is insufficient automation of interview data analysis and feedback, which can lead to delays in the evolution of interviewer training and evaluation criteria.

[1663] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1664] In this invention, the server includes means for generating a customized virtual interviewer based on user requests, means for conducting real-time dialogue with candidates and conducting Q&A sessions, means for collecting and analyzing interview data to provide an optimized interview experience, means for automatically learning the virtual interviewer's questioning style and evaluation criteria using user feedback and interview results, means for analyzing emotional evaluations, and means for adjusting questions based on the analysis results. This enables interactive interviews that take into account the candidate's emotional state, resulting in a high-quality interview experience and efficient talent evaluation.

[1665] "User" refers to an individual or organization that uses this system.

[1666] A "virtual interviewer" refers to a program that uses generative AI models and emotion engines to mimic the role of a real-life interviewer.

[1667] "Real-time interaction" refers to simultaneous interaction between the interviewer and candidate.

[1668] "Question and answer session" refers to the process in which the interviewer asks questions and the candidate answers them.

[1669] "Interview data" refers to information obtained during an interview, such as the candidate's responses, facial expressions, and tone of voice.

[1670] "Analysis" refers to the process of evaluating the collected interview data and deriving meaningful results.

[1671] An "optimized interview experience" refers to an interview process that is tailored to the candidate's emotional state and reactions.

[1672] "Feedback" refers to the evaluation and opinions provided by the user after the interview is completed.

[1673] "Questioning style" refers to the format or pattern of questions used by the interviewer.

[1674] "Evaluation criteria" refers to the standards or criteria for evaluating a candidate's responses or performance.

[1675] "Emotional assessment" refers to the process of analyzing a candidate's facial expressions and tone of voice to determine their emotional state.

[1676] "Analysis results" refer to the specific conclusions or findings obtained after conducting an emotion assessment.

[1677] "Question tailoring" refers to the process of changing or selecting the next question based on the candidate's emotional state.

[1678] This invention describes a specific implementation method of a virtual interviewer system that combines a user, a terminal, a server, and an emotion engine. This system generates a customized virtual interviewer based on the user's request and has functions such as real-time Q&A, analysis and provision of interview data, and optimization of questioning style based on feedback. In addition, by incorporating an emotion engine, it evaluates the candidate's emotional state and provides human-like interaction and comprehensive evaluation.

[1679] System Configuration

[1680] 1. User:

[1681] The user inputs a customization request for generating an interviewer corresponding to a specific industry and job type through the operation screen of the terminal.

[1682] 2. Terminal:

[1683] The terminal collects customization requests from users and sends them to a server. Wearable devices such as smart glasses are used for training and interviewing new operators.

[1684] 3. Server:

[1685] The server uses a generative AI model based on the user's input data to generate a virtual interviewer profile, which includes question patterns and evaluation criteria appropriate for the industry and job type. The generated virtual interviewer profile is saved by the server and prepared for the interview.

[1686] Real-time Q&A

[1687] The server activates the virtual interviewer at the start of the interview, and also activates the emotion engine. The server asks the first question through the virtual interviewer, and the candidate inputs their response through their terminal. The terminal sends this response to the server, and the emotion engine analyzes the candidate's facial expressions, tone of voice, choice of words, etc. to evaluate their emotional state.

[1688] For example, if a candidate answers, "I have experience operating machines," the emotion engine evaluates whether they are speaking with confidence based on their tone of voice and facial expressions. The server takes this analysis into account and generates the next question in real time. For example, if the emotion engine determines that the candidate is nervous, the server will present a question that will help them relax, such as, "Tell us about your specific project experience."

[1689] Learning Function

[1690] After the interview, the user enters feedback through the system. The device collects the feedback and sends it to the server. The server analyzes the collected feedback and uses it to optimize the questioning style and evaluation criteria. For example, if feedback is provided that "there are not enough questions to evaluate logical thinking," the server will reflect this in the AI ​​model and ask more detailed questions in the next interview.

[1691] Data analysis and provision

[1692] All data from the interview session is collected and stored by the server, including the candidate's responses, evaluation scores, collected sentiment data, and feedback. The server analyzes this data and provides insights to the user. For example, it analyzes the average technical skill scores of multiple candidates and trends in specific skill deficiencies, and provides this to the user as a report.

[1693] Providing flexible and scalable solutions

[1694] The server dynamically adjusts system resources according to the company's recruitment volume. For example, if a large number of interviews are being conducted at once, the server can automatically increase computing resources and distribute the load to ensure stable operation of the terminals.

[1695] Examples and prompts

[1696] For example, if a new factory operator answers, "I would like to learn how to operate a new machine," during training for new employees, the virtual interviewer will analyze their tone of voice and facial expression, and if it determines that they are nervous, it will put them at ease by asking them the next question, "Please explain the specific operating procedures."

[1697] Example prompt sentence:

[1698] You are a virtual training assistant helping train new employees in a factory. Based on the responses provided by the candidate, you use an emotion engine to analyze their emotional state and tailor the next questions you ask.

[1699] This enables interactive interviews that take into account the candidate's emotional state, resulting in a high-quality interview experience and efficient talent evaluation.

[1700] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1701] Step 1:

[1702] The user operates the device to input customization requests to generate a virtual interviewer for a specific industry and job type. The data input by the user includes the skills (e.g., programming skills, knowledge of machine operations) and characteristics (e.g., problem-solving ability, communication ability) to be evaluated. The input data is sent to the server in JSON format.

[1703] Step 2:

[1704] The device collects customization requests from the user and sends them to the server. The data sent as the customization request includes the user's input information. Upon receiving this data, the server uses a generative AI model to generate a profile for the virtual interviewer. The profile includes question patterns and evaluation criteria appropriate for the industry and job type.

[1705] Step 3:

[1706] The server saves the generated virtual interviewer profile and prepares for the interview. The profile saves specific questions and evaluation criteria in template format. The saved profile is used in subsequent processing.

[1707] Step 4:

[1708] After the interview date is arranged, the terminal sends an interview invitation to the candidate, and prepares the candidate so that they can log in at the specified date and time. The server verifies the login information and activates the virtual interviewer at the start time of the interview.

[1709] Step 5:

[1710] When the interview begins, the server asks the first question via the virtual interviewer, and the candidate enters their response via the terminal. The entered response data is sent in text format to the server, which then receives the candidate's response and analyzes the data using a natural language processing engine.

[1711] Step 6:

[1712] The server receives the candidate's responses and uses an emotion engine to analyze the candidate's facial expressions, tone of voice, and word choice. Based on the analysis results, the server evaluates the candidate's emotional state (e.g., nervous, confident, confused). The evaluation data is used to generate the next questions in real time.

[1713] Step 7:

[1714] Taking into account the analysis results of the emotion engine, the server generates the next question in real time. For example, if the server determines that the candidate is nervous, it will present a question that will help them relax, such as "Tell us about your specific project experience." The generated question is displayed to the candidate via their device.

[1715] Step 8:

[1716] The question and answer process is repeated until all questions are completed. The server records all collected data and stores all data from the interview session, including the candidate's responses, evaluation scores, collected emotional data, and feedback.

[1717] Step 9:

[1718] After the interview, the user enters feedback through the system. The terminal collects the feedback and sends it to the server. The server analyzes the collected feedback and optimizes the question style and evaluation criteria for the next interview, thereby improving the accuracy and effectiveness of the virtual interviewer.

[1719] Step 10:

[1720] The server generates an interview report based on the analysis results and provides it to the user. The report includes the average technical skill scores of multiple candidates, trends in specific skill deficiencies, and an overall evaluation of the interview. Users can use this report to optimize their recruitment activities.

[1721] The above are the specific processing steps for implementing the present invention, which enable interactive interviews that take into account the candidate's emotional state, thereby achieving a high-quality interview experience and efficient talent evaluation.

[1722] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1723] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1724] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1725] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1726] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1727] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1728] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1729] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1730] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1731] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1732] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present dis...

Claims

1. means for generating a customized virtual interviewer based on a user's requirements; A means to conduct real-time dialogue with candidates and hold Q&A sessions; A means to collect and analyze interview data to provide an optimized interview experience; A means for automatically learning the questioning style and evaluation criteria of the virtual interviewer using user feedback and interview results; A system including:

2. The system of claim 1 , wherein the generation of the customized virtual interviewer is appropriately adjusted based on industry, job type, and evaluation skills.

3. The system of claim 1 , wherein the virtual interviewer provides a means for assessing a candidate's skills and attributes along with real-time responses to questions.

Images