System

Abstract

A system is provided.SOLUTION: A system, comprising: means for collecting applicant paperwork; means for analyzing the collected paperwork and utilizing a generative AI model to evaluate the applicant's skills, experience, and educational background; means for generating and displaying evaluation results in the form of a visual report; means for utilizing the generated evaluation results to generate a set of interview questions; and means for integrating the paperwork examination and interview results, performing a final evaluation, and recommending informal candidates.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In traditional recruitment processes, subjectivity and unconscious bias on the part of judges during document screening and interviews have been major issues. This bias can lead to the inappropriate exclusion of certain candidates or to unfair selection. Furthermore, the lack of objective evaluation criteria makes it difficult for diverse talent to receive equal opportunities. To solve this problem, it is necessary to eliminate bias throughout the entire recruitment process and conduct fair and objective evaluations. [Means for solving the problem]

[0005] The present invention provides a means for collecting document data from applicants and analyzing it using a generative AI model. It also provides an evaluation means for fairly evaluating skills, experience, and educational background based on the analyzed data. It also provides a means for generating a personalized set of interview questions based on the results of these evaluations, and includes a means for analyzing voice and text data collected during interviews in real time to provide feedback. Finally, it provides a system for achieving fair and objective evaluation by integrating the results of the document review and interview, making a final evaluation, and recommending candidates for employment.

[0006] "Applicant" refers to an individual who applies for a particular job and undergoes document screening and interview.

[0007] "Document data" refers to a collection of information submitted by an applicant, such as a resume, curriculum vitae, or application form.

[0008] "Analysis" refers to the process of evaluating collected document data using a generative AI model.

[0009] A "generative AI model" refers to an algorithm that uses machine learning and natural language processing techniques to analyze data and generate evaluation results.

[0010] "Skills" refer to the abilities and techniques required to perform a specific job or function.

[0011] "Experience" refers to achievements and history related to past jobs and duties.

[0012] "Educational background" refers to information related to the applicant's education, such as their academic history, qualifications, and research activities.

[0013] "Evaluation results" refers to the comprehensive evaluation based on skills, experience, and educational background generated by the analysis.

[0014] "Report format" refers to the document or graph format used to visually display the evaluation results.

[0015] An "interview question set" refers to a set of questions generated for use during an interview.

[0016] "Real-time" refers to the process of analyzing data and providing feedback instantly during the interview.

[0017] "Integration" refers to the process of comprehensively evaluating the results of document review and interviews.

[0018] "Final evaluation" refers to the result of a comprehensive assessment of the document review and interview results.

[0019] "Offer candidate" refers to an applicant who is deemed appropriate to receive an offer based on the final evaluation.

[0020] "Recommendation" refers to the act of presenting a candidate for employment to HR personnel and using them as a reference for the final decision. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This system consists of multiple elements, including a server, a user terminal, and a generative AI model. Each element and its operation are explained in detail below.

[0043] System Configuration

[0044] 1. Server

[0045] The server acts as the central control unit and performs all data processing and analysis.

[0046] The server collects applicants' document data and analyzes it using a generative AI model.

[0047] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[0048] The server analyzes the data collected during the interview in real time and provides feedback.

[0049] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[0050] 2. User Device

[0051] The user terminal is a device used by HR personnel and interviewers.

[0052] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[0053] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[0054] 3. Generative AI Models

[0055] The generative AI model is implemented on the server and is used to analyze applicant document data.

[0056] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[0057] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[0058] Program processing

[0059] 1. User enters job information

[0060] Users input job information from their own devices and send it to the server, which then stores the received job information in a database.

[0061] 2. Collection and analysis of application documents

[0062] When an applicant submits a resume and work history, the server receives them and stores them in a database.

[0063] The server uses a generative AI model to analyze these documents and evaluate the applicant's skills, experience, and educational background.

[0064] The server generates an evaluation result based on the analysis result and transmits it to the user terminal.

[0065] 3. Generating a set of interview questions

[0066] The server generates an individually appropriate set of interview questions based on the analysis results.

[0067] The server transmits the generated interview question set to the user terminal.

[0068] 4. Conducting interviews and real-time evaluations

[0069] The user conducts an interview using the interview question set and transmits the voice data and text data collected during the interview to the server.

[0070] The server analyzes this data in real time and evaluates the candidate's communication skills and responses.

[0071] The server sends feedback to the user terminal in real time to support the evaluation procedure.

[0072] 5. Final evaluation and recommendation of job offers

[0073] The server will combine the results of the document review and interview and make a final evaluation.

[0074] The server selects job offer candidates based on the final evaluation results and transmits a recommendation list to the user terminal.

[0075] The user reviews the final candidates and makes a final decision.

[0076] Specific examples

[0077] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation results, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on these questions, and by sending data during the interview to the server, the server performs real-time analysis and provides feedback. Finally, the server integrates all the evaluation results, generates a recommended list of candidates for employment, and sends it to the user's device, thereby supporting fair hiring practices.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] User enters job information

[0081] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own terminals.

[0082] The user terminal transmits the input information to the server.

[0083] Step 2:

[0084] The server collects the applications.

[0085] The server receives document data (resume, curriculum vitae, etc.) submitted by applicants and stores it in a database.

[0086] Step 3:

[0087] The server analyzes the document data

[0088] The server inputs the application documents into a generative AI model and uses natural language processing technology to analyze skills, experience, and educational background.

[0089] Step 4:

[0090] The server generates the evaluation results

[0091] The server generates a skill matching score, an aptitude score, and an overall evaluation from the analysis results.

[0092] The server transmits the generated evaluation results in a visualized report format to the terminal of the HR staff.

[0093] Step 5:

[0094] The server generates a set of interview questions

[0095] The server generates an individually appropriate set of interview questions based on the evaluation results.

[0096] The server transmits the generated interview question set to the user terminal.

[0097] Step 6:

[0098] User conducts interview

[0099] The user conducts an interview based on the question set sent from the server.

[0100] Step 7:

[0101] The user sends the interview data to the server

[0102] The user transmits the voice data and text data collected during the interview to the server in real time.

[0103] Step 8:

[0104] The server analyzes the interview data in real time

[0105] The server analyzes the received voice and text data and evaluates communication skills and responses.

[0106] Step 9:

[0107] The server provides feedback to the user device

[0108] The server generates real-time feedback during or after the interview and sends it to the HR representative's terminal.

[0109] Step 10:

[0110] The server performs the final evaluation

[0111] The server integrates the document review and interview results and calculates an overall evaluation score for each candidate.

[0112] Step 11:

[0113] The server selects job candidates and generates a recommendation list

[0114] The server selects job offer candidates based on the final evaluation scores.

[0115] The server transmits a recommended list of job offer candidates to the terminal of the HR person.

[0116] Step 12:

[0117] User makes final decision and sends offer letter

[0118] The user makes a final decision based on the recommended list of job candidates.

[0119] The user generates and sends a job offer notice to the determined job offer candidate.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] In traditional hiring processes, it is difficult to accurately evaluate an applicant's skills and experience, and preparing interview questions requires a great deal of effort. Furthermore, evaluations during interviews tend to be subjective, making it difficult to ensure fair hiring practices. Furthermore, integrating multiple data sets to make a final evaluation requires a great deal of time and expertise.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for saving the analysis results in a database and generating and displaying evaluation results in the form of a visual report, means for automatically generating a set of interview questions based on the analysis results, means for analyzing data collected during interviews in real time and providing feedback, and means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment. This enables accurate evaluation of applicants' skills and experience, generation of fair interview questions, real-time evaluation feedback, and comprehensive evaluation integrating multiple data.

[0125] "Applicant's document data" refers to information on documents submitted by applicants, such as resumes and job history documents.

[0126] "Means of collection" refers to the methods and functions for obtaining document data submitted by applicants and storing it within the system.

[0127] "Analyzing" refers to the detailed evaluation and analysis of collected document data using data processing techniques and generative AI models.

[0128] "Generative AI model" refers to an artificial intelligence algorithm or technology used to evaluate an applicant's skills, experience, educational background, etc.

[0129] "Evaluation results" refers to the analysis results of an applicant's skills, experience, and educational background analyzed using a generative AI model.

[0130] "Means for displaying in a visual report format" refers to a method or function for converting the evaluation results into a visually easy-to-understand format such as a graph or chart and displaying them to the user.

[0131] An "interview question set" refers to a list of questions to be used during an interview, created based on the applicant's evaluation results.

[0132] "Real-time" analysis refers to the immediate processing and evaluation of audio and text data collected during interviews.

[0133] "Feedback" refers to evaluations and suggestions provided in real time based on the analysis results.

[0134] "Means for recommending candidates for employment" refers to the method and function of selecting and recommending appropriate candidates for employment based on a comprehensive evaluation of document screening and interview results.

[0135] MODE FOR CARRYING OUT THE INVENTION

[0136] This invention is a system that uses a server, a user terminal, and a generative AI model to efficiently evaluate applicant document data and support interviews. This will be explained in detail below.

[0137] Hardware and Software Configuration

[0138] 1. Server:

[0139] The server acts as a central control unit, collecting applicant document data and analyzing the data using a generative AI model.

[0140] The server is equipped with a database that stores received job information, applicant document data, and analysis results.

[0141] Based on the analysis results, a set of interview questions is generated and sent to the user's terminal.

[0142] 2. User Device:

[0143] The user terminal is a device used by HR personnel and interviewers, and is used to input job information, check evaluation results, receive interview question sets, send interview results, and so on.

[0144] It provides tools to receive feedback from the server in real time and conduct interviews effectively.

[0145] 3. Generative AI Model:

[0146] The generative AI model is implemented on a server and uses natural language processing technology to analyze applicants' skills, experience, and educational background.

[0147] A set of interview questions is automatically generated based on the analysis results.

[0148] Specific processing of the program

[0149] Collection and analysis of application documents

[0150] Resumes and work histories received from applicants are uploaded to the server in PDF or Word format. The server then uses OCR (Optical Character Recognition) technology to convert these documents into text data and stores it in a database. The generative AI model then analyzes the applicant's skills, experience, and educational background. The analysis results are stored in the database in numerical and graphical format and displayed on the user's device.

[0151] Generating a set of interview questions

[0152] Based on the analysis results of the generative AI model, the server automatically generates a set of interview questions suited to each applicant. For example, personalized questions are created based on the applicant's technical skills and past experience. This question set is sent to the user's device and can be used to help prepare for the interview.

[0153] Real-time interview evaluation and feedback

[0154] The user conducts an interview with an applicant using a set of interview questions and records audio and text data. This data is sent to the server in real time and analyzed by the generative AI model. The analysis results are fed back to the user's device in real time, providing an evaluation of communication skills and the quality of responses, thereby supporting the progress of the interview.

[0155] Final evaluation and recommendation of job candidates

[0156] The server integrates the results of the applicant's document review and interview evaluation to make a final evaluation. Based on the final evaluation results, a recommended list of candidates for employment is generated and sent to the user's terminal. The user then decides on the final candidates for employment based on this recommendation list.

[0157] Specific examples

[0158] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation result, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on this, and by sending data during the interview to the server, the server performs real-time analysis and provides feedback. Finally, the server integrates all the evaluation results and generates a recommended list of candidates for employment, supporting fair hiring practices.

[0159] Prompt Sentence Examples

[0160] "Enter your software engineer job posting and we'll collect and analyze your applications."

[0161] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0162] Step 1:

[0163] The user enters job information. The user uses a dedicated web form to enter job information such as job description, required skills, years of experience, etc., and sends it to the server. The server stores the received job information in a database.

[0164] Input: Job information (job description, required skills, years of experience, etc.)

[0165] Output: Job listings stored in a database

[0166] Specific operation: The server receives the job information as text data and stores it in the database using an SQL query.

[0167] Step 2:

[0168] The user collects the applicant's document data and sends it to the server. The applicant's resume and work history are uploaded in PDF or Word format. The server receives these files and stores them in a database.

[0169] Input: Resume, work history (PDF or Word format)

[0170] Output: Document data stored in a database

[0171] Specific operation: The server receives the uploaded file and stores it in a database along with the file name and applicant information.

[0172] Step 3:

[0173] The server analyzes the collected document data with a generative AI model to evaluate the applicant's skills, experience, and educational background. It uses OCR technology to convert the data into text. The generative AI model then analyzes the data using natural language processing technology.

[0174] Input: Document data (converted to text format)

[0175] Output: Analysis results (evaluation of skills, experience, and educational background)

[0176] How it works: The server uses OCR technology to extract text data from image files, then inputs the text data into a generative AI model for analysis. The analysis results are stored in a database.

[0177] Step 4:

[0178] The server generates a set of interview questions based on the analysis results.The server generates interview questions personalized for each applicant based on the analysis results.

[0179] Input: Analysis results (evaluation of skills, experience, and educational background)

[0180] Output: A set of interview questions

[0181] Specific operation: The server uses the generative AI model to generate a list of questions based on the analysis results, and the generated questions are stored in a database and sent to the user's device.

[0182] Step 5:

[0183] The user conducts an interview using a set of interview questions. The collected voice and text data is sent to a server and analyzed in real time. The server then analyzes the data using a generative AI model and provides feedback.

[0184] Input: Audio and text data collected during the interview

[0185] Output: Real-time feedback

[0186] Specific operation: The user conducts an interview and sends voice and text data to the server, which then analyzes this data in real time using a generative AI model and sends feedback to the user's device.

[0187] Step 6:

[0188] The server integrates the document screening and interview results, performs a final evaluation, and recommends candidates for employment. Based on the results of the final evaluation, the server creates a list of candidates for employment and sends the recommendation list to the user's terminal.

[0189] Input: Document review results, interview results

[0190] Output: Recommended list of job offers

[0191] Specific operation: The server integrates the document screening results and interview results, performs a comprehensive evaluation using a generative AI model, generates a list of job-success candidates based on the final evaluation, and sends it to the user's device.

[0192] (Application example 1)

[0193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] Conventional hiring processes have problems such as requiring a great deal of time and effort for document review of applicants, preparation of interview questions, and evaluation during interviews. Furthermore, real-time feedback is difficult, leading to inconsistent evaluation criteria for applicants. Large retail chains, in particular, often require an efficient and consistent hiring process. Therefore, the present invention was designed to solve these problems.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0196] In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for generating evaluation results and displaying them in a visual report format, means for using the generated evaluation results to generate an interview question set, means for analyzing data collected during the interview in real time and providing feedback, means for integrating the document screening and interview results, making a final evaluation, and recommending a candidate for employment, and means for displaying the interview question set and receiving real-time feedback through smart glasses, thereby improving the efficiency and consistency of applicant document screening and interviews and enabling real-time evaluation and feedback.

[0197] "Applicant document data" refers to a collection of information such as resumes and job histories submitted by job applicants, which contain information about an individual's skills, experience, educational background, etc.

[0198] A "generative AI model" is a computer program based on artificial intelligence technology that uses natural language processing and machine learning to evaluate applicants' skills, experience, educational background, etc.

[0199] A "visual report format" is a method of presenting analytical results in a visual format such as graphs, tables, or charts.

[0200] An "interview question set" is a set of questions to be used during an interview, generated based on the applicant's evaluation results.

[0201] "Analyzing data in real time" means instantly processing the audio and text data collected during the interview and obtaining the analysis results.

[0202] "Means of providing feedback" refers to a function that provides evaluations and advice to interviewers in real time during the interview based on the analysis results.

[0203] The "means of recommending candidates for job offers" refers to a method of integrating the results of document screening and interviews, conducting a final evaluation, and selecting and notifying suitable applicants for job offers.

[0204] "Smart glasses" are wearable devices equipped with a display and voice recognition functions, and are glasses-type terminals that can display information and receive instructions.

[0205] "Real-time feedback" refers to the instantaneous provision of data analysis results during the interview, providing evaluation information that the interviewer can use as a reference on the spot.

[0206] This invention is a system that collects, analyzes, and evaluates applicants' document data, generates interview question sets, provides real-time evaluation, feedback, and final evaluation. In particular, it uses smart glasses to streamline the interview process and provide real-time feedback.

[0207] System Configuration

[0208] 1. Server

[0209] Hardware: High-performance server unit (e.g., Amazon Web Services EC2 instance)

[0210] Software: Generative AI models running on servers, database systems (e.g., MySQL), and speech analysis software (e.g., Google Speech-to-Text API)

[0211] Functions: Document data collection, analysis by generative AI model, visual display of evaluation results, generation of interview question sets, real-time analysis and feedback, final evaluation and recommendation of job offers

[0212] 2. User Device

[0213] Hardware: Smart glasses (e.g. Google Glass), PC or tablet

[0214] Software: Dedicated application installed on the smart glasses

[0215] Functions: Enter job information, view interview question sets, receive real-time feedback, submit interview results, and check final evaluation results

[0216] System Operation

[0217] 1. Enter job information

[0218] The user uses the smart glasses to input job information by voice or by selecting from a simple menu, and then sends the information to the server.

[0219] 2. Collection and analysis of application documents

[0220] The server receives document data such as resumes and work histories submitted by applicants and stores them in a database. The server then analyzes the documents using a generative AI model (e.g., a natural language processing model such as BERT or GPT-3) to evaluate the applicant's skills, experience, and educational background.

[0221] 3. Generating and displaying interview question sets

[0222] Based on the analysis results, the server generates a set of interview questions and displays them on the user's smart glasses. This set of questions is personalized based on the applicant's evaluation results.

[0223] 4. Real-time evaluation

[0224] The voice and text data collected during the interview is sent from the smart glasses to a server and analyzed in real time using a generative AI model and voice analysis software, with the analysis results provided as immediate feedback to the smart glasses.

[0225] 5. Final evaluation and recommendation of job offers

[0226] The server integrates the applicant's document screening and interview results to make a final evaluation. Based on the evaluation results, it selects candidates for employment and sends a recommendation list to the user's device.

[0227] Specific examples

[0228] For example, when an HR person at a retail chain is looking to recruit a new cashier, they use smart glasses to enter the job information into a server. The server receives and analyzes the applicant's document data and generates an evaluation result. Based on the generated evaluation result, a set of interview questions is created and displayed on the HR person's smart glasses during the interview. Data analysis and real-time feedback are provided during the interview, and finally, the server integrates all the data to make a final evaluation and recommend candidates for employment.

[0229] Prompt Sentence Examples

[0230] "You are using smart glasses to interview applicants. What features do you want from smart glasses?"

[0231] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0232] Step 1:

[0233] Users use the smart glasses to input job information by voice or menu selection, and then send it to the server. The input is collected as voice data and converted into text using speech recognition software such as the Google Speech-to-Text API. The converted job information is then sent to the server and stored in a database.

[0234] Input: Audio data

[0235] Output: Text data of job information

[0236] Step 2:

[0237] The server receives document data such as resumes and curriculum vitae submitted by applicants and stores them in a database. The server then analyzes these documents using a generative AI model (e.g., BERT or GPT-3) to evaluate the applicants' skills, experience, and educational background.

[0238] Input: Applicant's document data

[0239] Output: Analysis results (evaluation data)

[0240] Step 3:

[0241] The server generates a set of interview questions based on the analysis results. The generative AI model takes into account the applicant's evaluation results and creates an optimal set of questions. The set of questions is then displayed on the user's smart glasses.

[0242] Input: Analysis results (evaluation data)

[0243] Output: Interview question set

[0244] Step 4:

[0245] The user conducts an interview based on a set of interview questions displayed on the smart glasses. During the interview, the smart glasses collect the applicant's voice and text data and transmit it to a server in real time. The server then analyzes the collected data using voice analysis software and a generative AI model to evaluate the applicant's communication skills and answers.

[0246] Input: Voice data and text data during the interview

[0247] Output: Real-time analysis results

[0248] Step 5:

[0249] The server generates feedback during the interview based on the results of real-time analysis and immediately transmits the feedback to the user's smart glasses, which the user can use to progress and evaluate the interview.

[0250] Input: Real-time analysis results

[0251] Output: Real-time feedback

[0252] Step 6:

[0253] The server integrates the collected document data and interview results to make a final evaluation. Based on this data, it selects candidates for employment and generates a recommendation list. The final evaluation and recommendation list are sent to the user's smart glasses for viewing.

[0254] Input: Document data, interview results

[0255] Output: Final evaluation results, recommendation list of job offers

[0256] Specific examples

[0257] For example, when a retail chain's HR staff is recruiting cashiers, they use smart glasses to input job information, receive document data from applicants, and analyze it. Based on the generated question set, they conduct interviews and evaluate candidates while receiving real-time feedback. Finally, the server integrates all the evaluation data and recommends candidates for employment.

[0258] Prompt Sentence Examples

[0259] "You are using smart glasses to interview applicants. What features do you want from smart glasses?"

[0260] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0261] This system consists of multiple elements, including a server, a user device, a generative AI model, and an emotion engine. Below, we will explain each element and its operation in detail.

[0262] System Configuration

[0263] 1. Server

[0264] The server acts as the central control unit and performs all data processing and analysis.

[0265] The server collects applicants' document data and analyzes it using a generative AI model.

[0266] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[0267] The server analyzes the data collected during the interview in real time and provides feedback.

[0268] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[0269] 2. User Device

[0270] The user terminal is a device used by HR personnel and interviewers.

[0271] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[0272] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[0273] 3. Generative AI Models

[0274] The generative AI model is implemented on the server and is used to analyze applicant document data.

[0275] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[0276] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[0277] 4. Emotion Engine

[0278] The emotion engine analyzes audio and video data collected during the interview to recognize the user's emotional state in real time.

[0279] The emotion engine generates and provides feedback to the user terminal based on the recognized emotional state.

[0280] The emotion engine has the function of visually displaying the emotional state.

[0281] Program processing

[0282] 1. User enters job information

[0283] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own devices and send it to the server, which then stores the received recruitment information in a database.

[0284] 2. Collection and analysis of application documents

[0285] When an applicant submits a resume and work history, the server receives them and stores them in a database.

[0286] The server uses a generative AI model to analyze these documents and evaluate the applicant's skills, experience, and educational background.

[0287] The server generates an evaluation result based on the analysis result and transmits it to the user terminal.

[0288] 3. Generating a set of interview questions

[0289] The server generates an individually appropriate set of interview questions based on the analysis results.

[0290] The server transmits the generated interview question set to the user terminal.

[0291] 4. Conducting interviews and real-time evaluations

[0292] The user conducts an interview using the interview question set and transmits the audio and video data collected during the interview to the server.

[0293] The server analyzes this data in real time and evaluates the candidate's communication skills and responses.

[0294] 5. Emotion Recognition by Emotion Engine

[0295] The emotion engine analyzes audio and video data during the interview and recognizes the user's emotional state in real time.

[0296] The recognized emotional state is reflected in the evaluation and is provided to the user terminal as feedback.

[0297] The feedback provides a visual display of emotional state that can be seen by the interviewer in real time.

[0298] 6. Final evaluation and recommendation of job offers

[0299] The server will combine the results of the document review and interview and make a final evaluation.

[0300] The server selects job offer candidates based on the final evaluation results and transmits a recommendation list to the user terminal.

[0301] The user reviews the final candidates and makes a final decision.

[0302] Specific examples

[0303] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation results, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on this and sends the data from the interview to the server, which performs real-time analysis and provides feedback. The emotion engine analyzes the audio and video data during the interview to recognize the user's emotional state in real time and reflects the results in the evaluation. Finally, the server integrates all the evaluation results, generates a recommended list of candidates, and sends it to the user's device, thereby supporting fair hiring.

[0304] The processing flow will be explained below.

[0305] Step 1:

[0306] User enters job information

[0307] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own terminals.

[0308] The user sends the entered information to the server.

[0309] The server stores the received job information in a database.

[0310] Step 2:

[0311] The server collects the applications.

[0312] Applicants submit resumes and work history.

[0313] The server receives these document data and stores them in a database.

[0314] Step 3:

[0315] The server analyzes the document data

[0316] The server inputs the collected document data into a generative AI model.

[0317] The generative AI model uses natural language processing technology to analyze applicants' skills, experience, and educational background.

[0318] Step 4:

[0319] The server generates the evaluation results

[0320] The server generates a skill matching score, an aptitude score, and an overall evaluation based on the analysis results of the generative AI model.

[0321] The server transmits the generated evaluation results in the form of a report to the user terminal.

[0322] Step 5:

[0323] The server generates a set of interview questions

[0324] The server generates an individually tailored set of interview questions based on the evaluation results.

[0325] The server transmits the generated interview question set to the user terminal.

[0326] Step 6:

[0327] User conducts interview

[0328] The user conducts an interview based on the question set sent from the server.

[0329] The user collects audio and video data during the interview and transmits it to the server.

[0330] Step 7:

[0331] The server analyzes the interview data in real time

[0332] The server analyzes the collected audio and video data in real time.

[0333] The server uses a generative AI model and an emotion engine to evaluate the applicant's communication skills and emotional state.

[0334] Step 8:

[0335] The server provides feedback to the user device

[0336] The server generates evaluation results in real time and transmits them to the user terminal.

[0337] Users adjust the progress of the interview based on real-time feedback.

[0338] Step 9:

[0339] The server visually displays the emotional state

[0340] Secondarily, the emotion engine generates data that visually represents the recognized emotional state.

[0341] The server transmits visual representation data of the emotional state to the user terminal.

[0342] The user terminal visually displays the emotional state as the interview progresses.

[0343] Step 10:

[0344] The server performs the final evaluation

[0345] The server integrates the document review and interview results and calculates an overall evaluation score for each applicant.

[0346] The server transmits the final evaluation result to the user terminal.

[0347] Step 11:

[0348] The server selects job candidates and generates a recommendation list

[0349] The server selects job offer candidates based on the final evaluation scores.

[0350] The server generates a recommendation list of job offer candidates and transmits it to the user terminal.

[0351] Step 12:

[0352] User makes final decision and sends offer letter

[0353] The user makes the final decision based on the recommendation list.

[0354] The user generates and sends a job offer notice to the selected job offer candidate.

[0355] Example 2

[0356] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] In conventional hiring processes, document screening and interview evaluation of applicants are heavily dependent on human subjectivity, making it difficult to ensure fairness and accuracy. It is also difficult to properly assess an applicant's emotional state and communication skills during the interview, creating a need for methods to improve the quality of interview evaluations. Furthermore, the personalization of interview question sets is insufficient, making it impossible to ask questions tailored to the characteristics of each applicant.

[0358] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for generating evaluation results and displaying them in a visual report format, means for using the generated evaluation results to generate an interview question set, means for analyzing data collected during interviews in real time and providing feedback, means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment, and means for analyzing audio and video data during interviews using an emotion engine to recognize the applicant's emotional state and reflect it in the evaluation. This enables objective evaluation of the applicant's skills and experience and real-time analysis of the applicant's emotional state and communication skills during the interview, improving the fairness and accuracy of the hiring process.

[0359] "Applicant's document data" refers to document information such as resumes and curriculum vitae submitted by applicants.

[0360] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to analyze data and evaluate applicants' skills, experience, and educational background.

[0361] "Displayed in a visual report format" refers to a method of presenting analytical results as visual information such as graphs and charts.

[0362] An "interview question set" refers to a series of questions used during an interview, personalized based on the applicant's evaluation results.

[0363] "Emotion engine" refers to a system that analyzes audio and video data and recognizes emotional states in real time.

[0364] "Feedback" refers to information or advice generated based on data analyzed during the interview.

[0365] "User terminal" refers to a device such as a computer or smartphone operated by a user.

[0366] "Real-time analytics" refers to analytics methods in which data is processed immediately as it is collected.

[0367] The system for implementing this invention is composed of multiple elements, including a server, a user terminal, a generative AI model, and an emotion engine. Each element and its operation will be specifically described below.

[0368] 1. Server

[0369] The server acts as the central control unit of the system and performs all data processing and analysis. The server mainly uses the following software and hardware:

[0370] A database management system (e.g., MySQL, PostgreSQL) is used to store applicant document data and job information.

[0371] Generative AI models (e.g., TensorFlow, PyTorch) are used to analyze applicants' resumes and CVs to assess their skills, experience, and educational background.

[0372] Using real-time analytics software, audio and video data collected during interviews is analyzed to assess candidates' communication skills and emotional state.

[0373] 2. User Device

[0374] User terminals are devices used by HR personnel and interviewers and have the following functions:

[0375] It provides an input interface for job information and transmits the input information to the server.

[0376] Receives and visually displays generated assessment results and interview question sets.

[0377] It has a communication function for sending data collected during the interview to a server in real time.

[0378] 3. Generative AI Models

[0379] The generative AI model is implemented on a server and analyzes applicant document data using the following techniques:

[0380] Using natural language processing techniques (e.g., BERT, GPT-3), we extract important information from applicant documents and evaluate that information.

[0381] A deep learning algorithm automatically generates a set of interview questions based on the analysis results.

[0382] 4. Emotion Engine

[0383] The Emotion Engine is a system that analyzes audio and video data collected during interviews to recognize emotional states in real time. This engine has the following functions:

[0384] Analyze voice data using voice recognition technology (e.g., Kaldi, DeepSpeech).

[0385] Using video analysis technology (e.g., OpenCV, Dlib), facial expressions and gestures are analyzed from video data to recognize emotional states.

[0386] The recognized emotional state is provided as feedback to the user terminal in real time, and the emotional state is visually displayed.

[0387] Specific examples

[0388] If a company posts a job opening for a "software engineer," the user enters the job information on their own device and sends it to the server. When the applicant submits a resume and work history, the server receives it, analyzes it using a generative AI model, and generates an evaluation result for the applicant. Based on the evaluation results, the server generates a personalized set of interview questions and sends them to the user's device. The user conducts an interview based on this question set, and audio and video data from the interview is sent to the server in real time. Using an emotion engine, the server analyzes this data, recognizes the user's emotional state in real time, and reflects it in the evaluation. Finally, the server integrates all the evaluation results and generates a recommended list of candidates for employment, which is sent to the user's device.

[0389] Prompt Sentence Examples

[0390] "Use this system to input a job posting for a 'Software Engineer' and analyze applicants' resumes and CVs. It also generates a set of interview questions and assesses the candidate's communication skills and emotional state in real time during the interview."

[0391] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0392] Step 1:

[0393] Users input company recruitment information from their own devices and send it to the server. The input recruitment information includes job type, required skills, years of experience, etc. The server saves the received recruitment information in a database. Specifically, it checks the format of the recruitment information and saves it in the database appropriately.

[0394] Input: Job information such as job type, required skills, years of experience, etc.

[0395] Data processing: Check the format of input information

[0396] Output: Job listings stored in a database

[0397] Step 2:

[0398] When an applicant submits a resume or work history, the server receives it and stores it in a database. The server checks the file format and stores it in the database appropriately.

[0399] Input: Applicant's resume and work history

[0400] Data processing: Checking file format and saving to database

[0401] Output: Document data stored in a database

[0402] Step 3:

[0403] The server analyzes the application documents using a generative AI model. Specifically, it uses natural language processing technology to extract the applicant's skills, experience, and educational background, and calculates evaluation points. Based on the analysis results, it generates evaluation results and sends them to the user's device.

[0404] Input: Saved applicant resume and CV

[0405] Data Calculation: Information Extraction and Evaluation Point Calculation Using Natural Language Processing

[0406] Output: Evaluation results sent to the user's device

[0407] Step 4:

[0408] The server generates an optimal interview question set for each applicant based on the analysis results. The generative AI model automatically selects relevant questions and creates a question set. The created question set is then sent to the user's device.

[0409] Input: Parsed evaluation results

[0410] Data Computation: Automatic Question Generation

[0411] Output: A set of interview questions sent to the user's terminal

[0412] Step 5:

[0413] The user conducts an interview based on the interview question set using a user terminal. Audio and video data are collected during the interview. The terminal transmits this data to the server in real time.

[0414] Input: Audio and video data collected during the interview

[0415] Data processing: Real-time collection and transmission of audio and video data

[0416] Output: Audio and video data sent to the server

[0417] Step 6:

[0418] The server analyzes the transmitted audio and video data in real time, uses an emotion engine to recognize the applicant's emotional state and reflect it in the evaluation, and generates feedback in real time and sends it to the user's device.

[0419] Input: Audio and video data received in real time

[0420] Data Computing: Analysis of Audio-Visual Data and Recognition of Emotional States

[0421] Output: Feedback sent to the user's device

[0422] Step 7:

[0423] The server combines the document screening evaluation results with the interview results to make a final evaluation. It weights each evaluation point and determines an overall evaluation. It generates a recommended list of candidates for employment and sends it to the user's terminal. The user then reviews the final candidates and makes a final decision.

[0424] Input: Document review evaluation results and interview results

[0425] Data calculation: Weighting of each evaluation point and determining the overall evaluation

[0426] Output: A list of recommended job candidates sent to the user's device

[0427] (Application example 2)

[0428] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0429] Conventional recruitment interview systems have difficulty effectively managing applicant document screening and interviews, particularly due to a lack of real-time feedback and recognition of emotional states. This prevents interviewers from fully understanding the true skills and emotional states of applicants, resulting in an inefficient hiring process and potentially unfair decisions. The present invention aims to solve these problems.

[0430] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for using the generated evaluation results to generate a set of interview questions, means for analyzing digital data collected during interviews in real time and providing feedback, means for analyzing collected audio and video data with an emotion recognition engine to recognize the applicant's emotional state in real time, means for allowing the interviewer to visually confirm the emotional state, and means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment. This makes it possible to grasp the applicant's skills and emotional state in real time and realize a fair and efficient hiring process.

[0431] "Applicant document data" refers to digital data that includes information such as resumes and curriculum vitae submitted by applicants.

[0432] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to evaluate an applicant's skills, experience, and educational background.

[0433] An "interview question set" is a list of questions that is individually created based on the evaluation results analyzed by the generative AI model and is used by interviewers with applicants.

[0434] The "emotion recognition engine" is a system that analyzes collected audio and video data and can recognize the emotional state of applicants in real time.

[0435] "Feedback" refers to assessments and advice provided in real time during the interview, which allows the interviewer to properly assess the candidate's skills and emotional state.

[0436] The "final evaluation" is a comprehensive evaluation that integrates the results of the document review and interviews, and serves as the basis for selecting candidates for employment.

[0437] An embodiment of the present invention is a system for collecting, analyzing, and evaluating document data from applicants, providing real-time feedback through interviews, and ultimately supporting hiring decisions. This system is composed of multiple elements, including a server, a user terminal, a generative AI model, and an emotion recognition engine.

[0438] System Configuration

[0439] 1. Server:

[0440] The server acts as the central control unit and performs all data processing and analysis.

[0441] The server collects applicants' document data and analyzes it using a generative AI model.

[0442] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[0443] The server analyzes the digital data collected during the interview in real time and provides feedback.

[0444] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[0445] 2. User Device:

[0446] A user terminal is a device used by a recruiter or interviewer.

[0447] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[0448] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[0449] 3. Generative AI Model:

[0450] The generative AI model is implemented on the server and is used to analyze applicant document data.

[0451] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[0452] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[0453] 4. Emotion Recognition Engine:

[0454] The emotion recognition engine analyzes audio and video data collected during the interview and recognizes the applicant's emotional state in real time.

[0455] The emotion recognition engine generates and provides feedback to the user terminal based on the recognized emotional state.

[0456] The emotion recognition engine has the function of visually displaying the emotional state.

[0457] Program processing explanation

[0458] 1. Data entry and storage:

[0459] Using a user terminal, a recruiter inputs job information into a server and stores it in a database.

[0460] 2. Document data collection and analysis:

[0461] Applicants submit their resumes and work histories via the application, which are then stored on the server.

[0462] The server uses a generative AI model to analyze these documents and generate a rating.

[0463] The results of this evaluation are used to generate a set of interview questions.

[0464] 3. Generate a set of interview questions:

[0465] The server uses a generative AI model to automatically generate a set of interview questions that are individually tailored.

[0466] This set of questions is sent to the recruiter's user terminal.

[0467] 4. Real-time analysis and feedback:

[0468] During the interview, the user's terminal transmits audio and video data to the server, which analyzes it in real time.

[0469] The server generates feedback based on this data and provides it to the user terminal.

[0470] 5. Emotion recognition:

[0471] The emotion recognition engine analyzes audio and video data to recognize the applicant's emotional state in real time.

[0472] The recognized emotional state is reflected in the evaluation and can be visually confirmed on the user's terminal.

[0473] 6. Final evaluation and recommendation:

[0474] The server will integrate the document review and interview results and make a final evaluation.

[0475] Based on the results of the final evaluation, we will recommend candidates for employment.

[0476] Specific examples

[0477] As an example, let's look at a use case in a store looking to hire a "software engineer." The store's recruiter would input the following prompts into the generative AI model on their smartphone:

[0478] Example prompt sentence:

[0479] "Analyze resumes, assess applicants' skills, years of experience, and educational background, and generate a set of interview questions, such as: 1. What role did you play in your most recent project? 2. Tell me about how you collaborate with your team. 3. Tell me more about the technology stack you used."

[0480] Based on this prompt, a generative AI model analyzes the applicant's documents and generates a set of interview questions that are individually tailored to the individual. The actual interview then takes place, and the server analyzes the data in real time, provides feedback, and visualizes the applicant's emotional state using an emotion recognition engine. Finally, the server integrates all the data, makes a final evaluation, and recommends suitable candidates for employment.

[0481] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0482] Step 1:

[0483] The user uses their own device to enter job information (job type, required skills, years of experience, etc.) into the corresponding application and sends it to the server. This data is stored in a database on the server. The input data includes details such as company name, position, required skills, etc. This allows the server to hold the job information necessary for subsequent processing.

[0484] Step 2:

[0485] Applicants upload their resumes, job history documents, and other documents to the server via the application. This data is stored in a database and used for further analysis. Specifically, documents in PDF and Word file formats are considered. This allows the server to collect detailed information about each applicant.

[0486] Step 3:

[0487] The server inputs the collected applicant document data into a generative AI model for analysis. The analysis uses natural language processing technology to extract the applicant's skills, experience, educational background, etc., and generates an evaluation result. This evaluation result includes details such as skill level, years of experience, and expertise. This allows for an objective evaluation of the applicant's suitability.

[0488] Step 4:

[0489] The server uses a generative AI model to automatically generate a set of interview questions based on the analyzed evaluation results. By inputting specific prompts, customized interview questions are generated for each applicant. For example, specific questions based on the applicant's technology stack or project experience can be included. This allows interviewers to prepare effective and relevant questions.

[0490] Step 5:

[0491] A set of interview questions is sent to the user's device, and the interviewer conducts the interview based on this. During the interview, audio and video data is sent from the user's device to the server in real time, providing input data for the server to analyze the collected data in real time.

[0492] Step 6:

[0493] The server analyzes the collected audio and video data in real time and generates feedback. An emotion recognition engine is used in the analysis to recognize the candidate's emotional state. Based on this, the server generates feedback and sends it to the user's device. This feedback includes an evaluation of the candidate's communication skills and emotional responses.

[0494] Step 7:

[0495] The emotion recognition engine recognizes the applicant's emotional state in real time based on the collected data. The recognition results are displayed visually on the user's device so that the interviewer can check them in real time. This display includes the type of emotion (e.g., joy, surprise, nervousness, etc.) and its intensity, making it easier for the interviewer to understand the applicant's emotional state.

[0496] Step 8:

[0497] After the interview, the server integrates the document review and interview results to make a final evaluation. This integration includes the applicant's skills, experience, interview performance, and emotion recognition results. Based on the final evaluation results, the server generates a list of recommended candidates and sends it to the user's device. This ensures fair and efficient hiring decisions.

[0498] Step 9:

[0499] Based on the final evaluation results, the user reviews the candidates and makes the final hiring decision. The final decision is sent from the user's device to the server and saved in the database. This records the entire hiring process and saves data that can be used for future reference and improvement.

[0500] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0501] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0502] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0503] [Second embodiment]

[0504] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0505] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0506] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0507] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0508] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0509] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0510] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0511] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0512] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0513] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0514] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0515] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0516] This system consists of multiple elements, including a server, a user terminal, and a generative AI model. Each element and its operation are explained in detail below.

[0517] System Configuration

[0518] 1. Server

[0519] The server acts as the central control unit and performs all data processing and analysis.

[0520] The server collects applicants' document data and analyzes it using a generative AI model.

[0521] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[0522] The server analyzes the data collected during the interview in real time and provides feedback.

[0523] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[0524] 2. User Device

[0525] The user terminal is a device used by HR personnel and interviewers.

[0526] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[0527] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[0528] 3. Generative AI Models

[0529] The generative AI model is implemented on the server and is used to analyze applicant document data.

[0530] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[0531] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[0532] Program processing

[0533] 1. User enters job information

[0534] Users input job information from their own devices and send it to the server, which then stores the received job information in a database.

[0535] 2. Collection and analysis of application documents

[0536] When an applicant submits a resume and work history, the server receives them and stores them in a database.

[0537] The server uses a generative AI model to analyze these documents and evaluate the applicant's skills, experience, and educational background.

[0538] The server generates an evaluation result based on the analysis result and transmits it to the user terminal.

[0539] 3. Generating a set of interview questions

[0540] The server generates an individually appropriate set of interview questions based on the analysis results.

[0541] The server transmits the generated interview question set to the user terminal.

[0542] 4. Conducting interviews and real-time evaluations

[0543] The user conducts an interview using the interview question set and transmits the voice data and text data collected during the interview to the server.

[0544] The server analyzes this data in real time and evaluates the candidate's communication skills and responses.

[0545] The server sends feedback to the user terminal in real time to support the evaluation procedure.

[0546] 5. Final evaluation and recommendation of job offers

[0547] The server will combine the results of the document review and interview and make a final evaluation.

[0548] The server selects job offer candidates based on the final evaluation results and transmits a recommendation list to the user terminal.

[0549] The user reviews the final candidates and makes a final decision.

[0550] Specific examples

[0551] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation results, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on these questions, and by sending data during the interview to the server, the server performs real-time analysis and provides feedback. Finally, the server integrates all the evaluation results, generates a recommended list of candidates for employment, and sends it to the user's device, thereby supporting fair hiring practices.

[0552] The processing flow will be explained below.

[0553] Step 1:

[0554] User enters job information

[0555] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own terminals.

[0556] The user terminal transmits the input information to the server.

[0557] Step 2:

[0558] The server collects the applications.

[0559] The server receives document data (resume, curriculum vitae, etc.) submitted by applicants and stores it in a database.

[0560] Step 3:

[0561] The server analyzes the document data

[0562] The server inputs the application documents into a generative AI model and uses natural language processing technology to analyze skills, experience, and educational background.

[0563] Step 4:

[0564] The server generates the evaluation results

[0565] The server generates a skill matching score, an aptitude score, and an overall evaluation from the analysis results.

[0566] The server transmits the generated evaluation results in a visualized report format to the terminal of the HR staff.

[0567] Step 5:

[0568] The server generates a set of interview questions

[0569] The server generates an individually appropriate set of interview questions based on the evaluation results.

[0570] The server transmits the generated interview question set to the user terminal.

[0571] Step 6:

[0572] User conducts interview

[0573] The user conducts an interview based on the question set sent from the server.

[0574] Step 7:

[0575] The user sends the interview data to the server

[0576] The user transmits the voice data and text data collected during the interview to the server in real time.

[0577] Step 8:

[0578] The server analyzes the interview data in real time

[0579] The server analyzes the received voice and text data and evaluates communication skills and responses.

[0580] Step 9:

[0581] The server provides feedback to the user device

[0582] The server generates real-time feedback during or after the interview and sends it to the HR representative's terminal.

[0583] Step 10:

[0584] The server performs the final evaluation

[0585] The server integrates the document review and interview results and calculates an overall evaluation score for each candidate.

[0586] Step 11:

[0587] The server selects job candidates and generates a recommendation list

[0588] The server selects job offer candidates based on the final evaluation scores.

[0589] The server transmits a recommended list of job offer candidates to the terminal of the HR person.

[0590] Step 12:

[0591] User makes final decision and sends offer letter

[0592] The user makes a final decision based on the recommended list of job candidates.

[0593] The user generates and sends a job offer notice to the determined job offer candidate.

[0594] Example 1

[0595] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0596] In traditional hiring processes, it is difficult to accurately evaluate an applicant's skills and experience, and preparing interview questions requires a great deal of effort. Furthermore, evaluations during interviews tend to be subjective, making it difficult to ensure fair hiring practices. Furthermore, integrating multiple data sets to make a final evaluation requires a great deal of time and expertise.

[0597] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0598] In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for saving the analysis results in a database and generating and displaying evaluation results in the form of a visual report, means for automatically generating a set of interview questions based on the analysis results, means for analyzing data collected during interviews in real time and providing feedback, and means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment. This enables accurate evaluation of applicants' skills and experience, generation of fair interview questions, real-time evaluation feedback, and comprehensive evaluation integrating multiple data.

[0599] "Applicant's document data" refers to information on documents submitted by applicants, such as resumes and job history documents.

[0600] "Means of collection" refers to the methods and functions for obtaining document data submitted by applicants and storing it within the system.

[0601] "Analyzing" refers to the detailed evaluation and analysis of collected document data using data processing techniques and generative AI models.

[0602] "Generative AI model" refers to an artificial intelligence algorithm or technology used to evaluate an applicant's skills, experience, educational background, etc.

[0603] "Evaluation results" refers to the analysis results of an applicant's skills, experience, and educational background analyzed using a generative AI model.

[0604] "Means for displaying in a visual report format" refers to a method or function for converting the evaluation results into a visually easy-to-understand format such as a graph or chart and displaying them to the user.

[0605] An "interview question set" refers to a list of questions to be used during an interview, created based on the applicant's evaluation results.

[0606] "Real-time" analysis refers to the immediate processing and evaluation of audio and text data collected during interviews.

[0607] "Feedback" refers to evaluations and suggestions provided in real time based on the analysis results.

[0608] "Means for recommending candidates for employment" refers to the method and function of selecting and recommending appropriate candidates for employment based on a comprehensive evaluation of document screening and interview results.

[0609] MODE FOR CARRYING OUT THE INVENTION

[0610] This invention is a system that uses a server, a user terminal, and a generative AI model to efficiently evaluate applicant document data and support interviews. This will be explained in detail below.

[0611] Hardware and Software Configuration

[0612] 1. Server:

[0613] The server acts as a central control unit, collecting applicant document data and analyzing the data using a generative AI model.

[0614] The server is equipped with a database that stores received job information, applicant document data, and analysis results.

[0615] Based on the analysis results, a set of interview questions is generated and sent to the user's terminal.

[0616] 2. User Device:

[0617] The user terminal is a device used by HR personnel and interviewers, and is used to input job information, check evaluation results, receive interview question sets, send interview results, and so on.

[0618] It provides tools to receive feedback from the server in real time and conduct interviews effectively.

[0619] 3. Generative AI Model:

[0620] The generative AI model is implemented on a server and uses natural language processing technology to analyze applicants' skills, experience, and educational background.

[0621] A set of interview questions is automatically generated based on the analysis results.

[0622] Specific processing of the program

[0623] Collection and analysis of application documents

[0624] Resumes and work histories received from applicants are uploaded to the server in PDF or Word format. The server then uses OCR (Optical Character Recognition) technology to convert these documents into text data and stores it in a database. The generative AI model then analyzes the applicant's skills, experience, and educational background. The analysis results are stored in the database in numerical and graphical format and displayed on the user's device.

[0625] Generating a set of interview questions

[0626] Based on the analysis results of the generative AI model, the server automatically generates a set of interview questions suited to each applicant. For example, personalized questions are created based on the applicant's technical skills and past experience. This question set is sent to the user's device and can be used to help prepare for the interview.

[0627] Real-time interview evaluation and feedback

[0628] The user conducts an interview with an applicant using a set of interview questions and records audio and text data. This data is sent to the server in real time and analyzed by the generative AI model. The analysis results are fed back to the user's device in real time, providing an evaluation of communication skills and the quality of responses, thereby supporting the progress of the interview.

[0629] Final evaluation and recommendation of job candidates

[0630] The server integrates the results of the applicant's document review and interview evaluation to make a final evaluation. Based on the final evaluation results, a recommended list of candidates for employment is generated and sent to the user's terminal. The user then decides on the final candidates for employment based on this recommendation list.

[0631] Specific examples

[0632] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation result, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on this, and by sending data during the interview to the server, the server performs real-time analysis and provides feedback. Finally, the server integrates all the evaluation results and generates a recommended list of candidates for employment, supporting fair hiring practices.

[0633] Prompt Sentence Examples

[0634] "Enter your software engineer job posting and we'll collect and analyze your applications."

[0635] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0636] Step 1:

[0637] The user enters job information. The user uses a dedicated web form to enter job information such as job description, required skills, years of experience, etc., and sends it to the server. The server stores the received job information in a database.

[0638] Input: Job information (job description, required skills, years of experience, etc.)

[0639] Output: Job listings stored in a database

[0640] Specific operation: The server receives the job information as text data and stores it in the database using an SQL query.

[0641] Step 2:

[0642] The user collects the applicant's document data and sends it to the server. The applicant's resume and work history are uploaded in PDF or Word format. The server receives these files and stores them in a database.

[0643] Input: Resume, work history (PDF or Word format)

[0644] Output: Document data stored in a database

[0645] Specific operation: The server receives the uploaded file and stores it in a database along with the file name and applicant information.

[0646] Step 3:

[0647] The server analyzes the collected document data with a generative AI model to evaluate the applicant's skills, experience, and educational background. It uses OCR technology to convert the data into text. The generative AI model then analyzes the data using natural language processing technology.

[0648] Input: Document data (converted to text format)

[0649] Output: Analysis results (evaluation of skills, experience, and educational background)

[0650] How it works: The server uses OCR technology to extract text data from image files, then inputs the text data into a generative AI model for analysis. The analysis results are stored in a database.

[0651] Step 4:

[0652] The server generates a set of interview questions based on the analysis results.The server generates interview questions personalized for each applicant based on the analysis results.

[0653] Input: Analysis results (evaluation of skills, experience, and educational background)

[0654] Output: A set of interview questions

[0655] Specific operation: The server uses the generative AI model to generate a list of questions based on the analysis results, and the generated questions are stored in a database and sent to the user's device.

[0656] Step 5:

[0657] The user conducts an interview using a set of interview questions. The collected voice and text data is sent to a server and analyzed in real time. The server then analyzes the data using a generative AI model and provides feedback.

[0658] Input: Audio and text data collected during the interview

[0659] Output: Real-time feedback

[0660] Specific operation: The user conducts an interview and sends voice and text data to the server, which then analyzes this data in real time using a generative AI model and sends feedback to the user's device.

[0661] Step 6:

[0662] The server integrates the document screening and interview results, performs a final evaluation, and recommends candidates for employment. Based on the results of the final evaluation, the server creates a list of candidates for employment and sends the recommendation list to the user's terminal.

[0663] Input: Document review results, interview results

[0664] Output: Recommended list of job offers

[0665] Specific operation: The server integrates the document screening results and interview results, performs a comprehensive evaluation using a generative AI model, generates a list of job-success candidates based on the final evaluation, and sends it to the user's device.

[0666] (Application example 1)

[0667] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0668] Conventional hiring processes have problems such as requiring a great deal of time and effort for document review of applicants, preparation of interview questions, and evaluation during interviews. Furthermore, real-time feedback is difficult, leading to inconsistent evaluation criteria for applicants. Large retail chains, in particular, often require an efficient and consistent hiring process. Therefore, the present invention was designed to solve these problems.

[0669] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0670] In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for generating evaluation results and displaying them in a visual report format, means for using the generated evaluation results to generate an interview question set, means for analyzing data collected during the interview in real time and providing feedback, means for integrating the document screening and interview results, making a final evaluation, and recommending a candidate for employment, and means for displaying the interview question set and receiving real-time feedback through smart glasses, thereby improving the efficiency and consistency of applicant document screening and interviews and enabling real-time evaluation and feedback.

[0671] "Applicant document data" refers to a collection of information such as resumes and job histories submitted by job applicants, which contain information about an individual's skills, experience, educational background, etc.

[0672] A "generative AI model" is a computer program based on artificial intelligence technology that uses natural language processing and machine learning to evaluate applicants' skills, experience, educational background, etc.

[0673] A "visual report format" is a method of presenting analytical results in a visual format such as graphs, tables, or charts.

[0674] An "interview question set" is a set of questions to be used during an interview, generated based on the applicant's evaluation results.

[0675] "Analyzing data in real time" means instantly processing the audio and text data collected during the interview and obtaining the analysis results.

[0676] "Means of providing feedback" refers to a function that provides evaluations and advice to interviewers in real time during the interview based on the analysis results.

[0677] The "means of recommending candidates for job offers" refers to a method of integrating the results of document screening and interviews, conducting a final evaluation, and selecting and notifying suitable applicants for job offers.

[0678] "Smart glasses" are wearable devices equipped with a display and voice recognition functions, and are glasses-type terminals that can display information and receive instructions.

[0679] "Real-time feedback" refers to the instantaneous provision of data analysis results during the interview, providing evaluation information that the interviewer can use as a reference on the spot.

[0680] This invention is a system that collects, analyzes, and evaluates applicants' document data, generates interview question sets, provides real-time evaluation, feedback, and final evaluation. In particular, it uses smart glasses to streamline the interview process and provide real-time feedback.

[0681] System Configuration

[0682] 1. Server

[0683] Hardware: High-performance server unit (e.g., Amazon Web Services EC2 instance)

[0684] Software: Generative AI models running on servers, database systems (e.g., MySQL), and speech analysis software (e.g., Google Speech-to-Text API)

[0685] Functions: Document data collection, analysis by generative AI model, visual display of evaluation results, generation of interview question sets, real-time analysis and feedback, final evaluation and recommendation of job offers

[0686] 2. User Device

[0687] Hardware: Smart glasses (e.g. Google Glass), PC or tablet

[0688] Software: Dedicated application installed on the smart glasses

[0689] Functions: Enter job information, view interview question sets, receive real-time feedback, submit interview results, and check final evaluation results

[0690] System Operation

[0691] 1. Enter job information

[0692] The user uses the smart glasses to input job information by voice or by selecting from a simple menu, and then sends the information to the server.

[0693] 2. Collection and analysis of application documents

[0694] The server receives document data such as resumes and work histories submitted by applicants and stores them in a database. The server then analyzes the documents using a generative AI model (e.g., a natural language processing model such as BERT or GPT-3) to evaluate the applicant's skills, experience, and educational background.

[0695] 3. Generating and displaying interview question sets

[0696] Based on the analysis results, the server generates a set of interview questions and displays them on the user's smart glasses. This set of questions is personalized based on the applicant's evaluation results.

[0697] 4. Real-time evaluation

[0698] The voice and text data collected during the interview is sent from the smart glasses to a server and analyzed in real time using a generative AI model and voice analysis software, with the analysis results provided as immediate feedback to the smart glasses.

[0699] 5. Final evaluation and recommendation of job offers

[0700] The server integrates the applicant's document screening and interview results to make a final evaluation. Based on the evaluation results, it selects candidates for employment and sends a recommendation list to the user's device.

[0701] Specific examples

[0702] For example, when an HR person at a retail chain is looking to recruit a new cashier, they use smart glasses to enter the job information into a server. The server receives and analyzes the applicant's document data and generates an evaluation result. Based on the generated evaluation result, a set of interview questions is created and displayed on the HR person's smart glasses during the interview. Data analysis and real-time feedback are provided during the interview, and finally, the server integrates all the data to make a final evaluation and recommend candidates for employment.

[0703] Prompt Sentence Examples

[0704] "You are using smart glasses to interview applicants. What features do you want from smart glasses?"

[0705] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0706] Step 1:

[0707] Users use the smart glasses to input job information by voice or menu selection, and then send it to the server. The input is collected as voice data and converted into text using speech recognition software such as the Google Speech-to-Text API. The converted job information is then sent to the server and stored in a database.

[0708] Input: Audio data

[0709] Output: Text data of job information

[0710] Step 2:

[0711] The server receives document data such as resumes and curriculum vitae submitted by applicants and stores them in a database. The server then analyzes these documents using a generative AI model (e.g., BERT or GPT-3) to evaluate the applicants' skills, experience, and educational background.

[0712] Input: Applicant's document data

[0713] Output: Analysis results (evaluation data)

[0714] Step 3:

[0715] The server generates a set of interview questions based on the analysis results. The generative AI model takes into account the applicant's evaluation results and creates an optimal set of questions. The set of questions is then displayed on the user's smart glasses.

[0716] Input: Analysis results (evaluation data)

[0717] Output: Interview question set

[0718] Step 4:

[0719] The user conducts an interview based on a set of interview questions displayed on the smart glasses. During the interview, the smart glasses collect the applicant's voice and text data and transmit it to a server in real time. The server then analyzes the collected data using voice analysis software and a generative AI model to evaluate the applicant's communication skills and answers.

[0720] Input: Voice data and text data during the interview

[0721] Output: Real-time analysis results

[0722] Step 5:

[0723] The server generates feedback during the interview based on the results of real-time analysis and immediately transmits the feedback to the user's smart glasses, which the user can use to progress and evaluate the interview.

[0724] Input: Real-time analysis results

[0725] Output: Real-time feedback

[0726] Step 6:

[0727] The server integrates the collected document data and interview results to make a final evaluation. Based on this data, it selects candidates for employment and generates a recommendation list. The final evaluation and recommendation list are sent to the user's smart glasses for viewing.

[0728] Input: Document data, interview results

[0729] Output: Final evaluation results, recommendation list of job offers

[0730] Specific examples

[0731] For example, when a retail chain's HR staff is recruiting cashiers, they use smart glasses to input job information, receive document data from applicants, and analyze it. Based on the generated question set, they conduct interviews and evaluate candidates while receiving real-time feedback. Finally, the server integrates all the evaluation data and recommends candidates for employment.

[0732] Prompt Sentence Examples

[0733] "You are using smart glasses to interview applicants. What features do you want from smart glasses?"

[0734] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0735] This system consists of multiple elements, including a server, a user device, a generative AI model, and an emotion engine. Below, we will explain each element and its operation in detail.

[0736] System Configuration

[0737] 1. Server

[0738] The server acts as the central control unit and performs all data processing and analysis.

[0739] The server collects applicants' document data and analyzes it using a generative AI model.

[0740] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[0741] The server analyzes the data collected during the interview in real time and provides feedback.

[0742] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[0743] 2. User Device

[0744] The user terminal is a device used by HR personnel and interviewers.

[0745] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[0746] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[0747] 3. Generative AI Models

[0748] The generative AI model is implemented on the server and is used to analyze applicant document data.

[0749] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[0750] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[0751] 4. Emotion Engine

[0752] The emotion engine analyzes audio and video data collected during the interview to recognize the user's emotional state in real time.

[0753] The emotion engine generates and provides feedback to the user terminal based on the recognized emotional state.

[0754] The emotion engine has the function of visually displaying the emotional state.

[0755] Program processing

[0756] 1. User enters job information

[0757] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own devices and send it to the server, which then stores the received recruitment information in a database.

[0758] 2. Collection and analysis of application documents

[0759] When an applicant submits a resume and work history, the server receives them and stores them in a database.

[0760] The server uses a generative AI model to analyze these documents and evaluate the applicant's skills, experience, and educational background.

[0761] The server generates an evaluation result based on the analysis result and transmits it to the user terminal.

[0762] 3. Generating a set of interview questions

[0763] The server generates an individually appropriate set of interview questions based on the analysis results.

[0764] The server transmits the generated interview question set to the user terminal.

[0765] 4. Conducting interviews and real-time evaluations

[0766] The user conducts an interview using the interview question set and transmits the audio and video data collected during the interview to the server.

[0767] The server analyzes this data in real time and evaluates the candidate's communication skills and responses.

[0768] 5. Emotion Recognition by Emotion Engine

[0769] The emotion engine analyzes audio and video data during the interview and recognizes the user's emotional state in real time.

[0770] The recognized emotional state is reflected in the evaluation and is provided to the user terminal as feedback.

[0771] The feedback provides a visual display of emotional state that can be seen by the interviewer in real time.

[0772] 6. Final evaluation and recommendation of job offers

[0773] The server will combine the results of the document review and interview and make a final evaluation.

[0774] The server selects job offer candidates based on the final evaluation results and transmits a recommendation list to the user terminal.

[0775] The user reviews the final candidates and makes a final decision.

[0776] Specific examples

[0777] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation results, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on this and sends the data from the interview to the server, which performs real-time analysis and provides feedback. The emotion engine analyzes the audio and video data during the interview to recognize the user's emotional state in real time and reflects the results in the evaluation. Finally, the server integrates all the evaluation results, generates a recommended list of candidates, and sends it to the user's device, thereby supporting fair hiring.

[0778] The processing flow will be explained below.

[0779] Step 1:

[0780] User enters job information

[0781] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own terminals.

[0782] The user sends the entered information to the server.

[0783] The server stores the received job information in a database.

[0784] Step 2:

[0785] The server collects the applications.

[0786] Applicants submit resumes and work history.

[0787] The server receives these document data and stores them in a database.

[0788] Step 3:

[0789] The server analyzes the document data

[0790] The server inputs the collected document data into a generative AI model.

[0791] The generative AI model uses natural language processing technology to analyze applicants' skills, experience, and educational background.

[0792] Step 4:

[0793] The server generates the evaluation results

[0794] The server generates a skill matching score, an aptitude score, and an overall evaluation based on the analysis results of the generative AI model.

[0795] The server transmits the generated evaluation results in the form of a report to the user terminal.

[0796] Step 5:

[0797] The server generates a set of interview questions

[0798] The server generates an individually tailored set of interview questions based on the evaluation results.

[0799] The server transmits the generated interview question set to the user terminal.

[0800] Step 6:

[0801] User conducts interview

[0802] The user conducts an interview based on the question set sent from the server.

[0803] The user collects audio and video data during the interview and transmits it to the server.

[0804] Step 7:

[0805] The server analyzes the interview data in real time

[0806] The server analyzes the collected audio and video data in real time.

[0807] The server uses a generative AI model and an emotion engine to evaluate the applicant's communication skills and emotional state.

[0808] Step 8:

[0809] The server provides feedback to the user device

[0810] The server generates evaluation results in real time and transmits them to the user terminal.

[0811] Users adjust the progress of the interview based on real-time feedback.

[0812] Step 9:

[0813] The server visually displays the emotional state

[0814] Secondarily, the emotion engine generates data that visually represents the recognized emotional state.

[0815] The server transmits visual representation data of the emotional state to the user terminal.

[0816] The user terminal visually displays the emotional state as the interview progresses.

[0817] Step 10:

[0818] The server performs the final evaluation

[0819] The server integrates the document review and interview results and calculates an overall evaluation score for each applicant.

[0820] The server transmits the final evaluation result to the user terminal.

[0821] Step 11:

[0822] The server selects job candidates and generates a recommendation list

[0823] The server selects job offer candidates based on the final evaluation scores.

[0824] The server generates a recommendation list of job offer candidates and transmits it to the user terminal.

[0825] Step 12:

[0826] User makes final decision and sends offer letter

[0827] The user makes the final decision based on the recommendation list.

[0828] The user generates and sends a job offer notice to the selected job offer candidate.

[0829] Example 2

[0830] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0831] In conventional hiring processes, document screening and interview evaluation of applicants are heavily dependent on human subjectivity, making it difficult to ensure fairness and accuracy. It is also difficult to properly assess an applicant's emotional state and communication skills during the interview, creating a need for methods to improve the quality of interview evaluations. Furthermore, the personalization of interview question sets is insufficient, making it impossible to ask questions tailored to the characteristics of each applicant.

[0832] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for generating evaluation results and displaying them in a visual report format, means for using the generated evaluation results to generate an interview question set, means for analyzing data collected during interviews in real time and providing feedback, means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment, and means for analyzing audio and video data during interviews using an emotion engine to recognize the applicant's emotional state and reflect it in the evaluation. This enables objective evaluation of the applicant's skills and experience and real-time analysis of the applicant's emotional state and communication skills during the interview, improving the fairness and accuracy of the hiring process.

[0833] "Applicant's document data" refers to document information such as resumes and curriculum vitae submitted by applicants.

[0834] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to analyze data and evaluate applicants' skills, experience, and educational background.

[0835] "Displayed in a visual report format" refers to a method of presenting analytical results as visual information such as graphs and charts.

[0836] An "interview question set" refers to a series of questions used during an interview, personalized based on the applicant's evaluation results.

[0837] "Emotion engine" refers to a system that analyzes audio and video data and recognizes emotional states in real time.

[0838] "Feedback" refers to information or advice generated based on data analyzed during the interview.

[0839] "User terminal" refers to a device such as a computer or smartphone operated by a user.

[0840] "Real-time analytics" refers to analytics methods in which data is processed immediately as it is collected.

[0841] The system for implementing this invention is composed of multiple elements, including a server, a user terminal, a generative AI model, and an emotion engine. Each element and its operation will be specifically described below.

[0842] 1. Server

[0843] The server acts as the central control unit of the system and performs all data processing and analysis. The server mainly uses the following software and hardware:

[0844] A database management system (e.g., MySQL, PostgreSQL) is used to store applicant document data and job information.

[0845] Generative AI models (e.g., TensorFlow, PyTorch) are used to analyze applicants' resumes and CVs to assess their skills, experience, and educational background.

[0846] Using real-time analytics software, audio and video data collected during interviews is analyzed to assess candidates' communication skills and emotional state.

[0847] 2. User Device

[0848] User terminals are devices used by HR personnel and interviewers and have the following functions:

[0849] It provides an input interface for job information and transmits the input information to the server.

[0850] Receives and visually displays generated assessment results and interview question sets.

[0851] It has a communication function for sending data collected during the interview to a server in real time.

[0852] 3. Generative AI Models

[0853] The generative AI model is implemented on a server and analyzes applicant document data using the following techniques:

[0854] Using natural language processing techniques (e.g., BERT, GPT-3), we extract important information from applicant documents and evaluate that information.

[0855] A deep learning algorithm automatically generates a set of interview questions based on the analysis results.

[0856] 4. Emotion Engine

[0857] The Emotion Engine is a system that analyzes audio and video data collected during interviews to recognize emotional states in real time. This engine has the following functions:

[0858] Analyze voice data using voice recognition technology (e.g., Kaldi, DeepSpeech).

[0859] Using video analysis technology (e.g., OpenCV, Dlib), facial expressions and gestures are analyzed from video data to recognize emotional states.

[0860] The recognized emotional state is provided as feedback to the user terminal in real time, and the emotional state is visually displayed.

[0861] Specific examples

[0862] If a company posts a job opening for a "software engineer," the user enters the job information on their own device and sends it to the server. When the applicant submits a resume and work history, the server receives it, analyzes it using a generative AI model, and generates an evaluation result for the applicant. Based on the evaluation results, the server generates a personalized set of interview questions and sends them to the user's device. The user conducts an interview based on this question set, and audio and video data from the interview is sent to the server in real time. Using an emotion engine, the server analyzes this data, recognizes the user's emotional state in real time, and reflects it in the evaluation. Finally, the server integrates all the evaluation results and generates a recommended list of candidates for employment, which is sent to the user's device.

[0863] Prompt Sentence Examples

[0864] "Use this system to input a job posting for a 'Software Engineer' and analyze applicants' resumes and CVs. It also generates a set of interview questions and assesses the candidate's communication skills and emotional state in real time during the interview."

[0865] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0866] Step 1:

[0867] Users input company recruitment information from their own devices and send it to the server. The input recruitment information includes job type, required skills, years of experience, etc. The server saves the received recruitment information in a database. Specifically, it checks the format of the recruitment information and saves it in the database appropriately.

[0868] Input: Job information such as job type, required skills, years of experience, etc.

[0869] Data processing: Check the format of input information

[0870] Output: Job listings stored in a database

[0871] Step 2:

[0872] When an applicant submits a resume or work history, the server receives it and stores it in a database. The server checks the file format and stores it in the database appropriately.

[0873] Input: Applicant's resume and work history

[0874] Data processing: Checking file format and saving to database

[0875] Output: Document data stored in a database

[0876] Step 3:

[0877] The server analyzes the application documents using a generative AI model. Specifically, it uses natural language processing technology to extract the applicant's skills, experience, and educational background, and calculates evaluation points. Based on the analysis results, it generates evaluation results and sends them to the user's device.

[0878] Input: Saved applicant resume and CV

[0879] Data Calculation: Information Extraction and Evaluation Point Calculation Using Natural Language Processing

[0880] Output: Evaluation results sent to the user's device

[0881] Step 4:

[0882] The server generates an optimal interview question set for each applicant based on the analysis results. The generative AI model automatically selects relevant questions and creates a question set. The created question set is then sent to the user's device.

[0883] Input: Parsed evaluation results

[0884] Data Computation: Automatic Question Generation

[0885] Output: A set of interview questions sent to the user's terminal

[0886] Step 5:

[0887] The user conducts an interview based on the interview question set using a user terminal. Audio and video data are collected during the interview. The terminal transmits this data to the server in real time.

[0888] Input: Audio and video data collected during the interview

[0889] Data processing: Real-time collection and transmission of audio and video data

[0890] Output: Audio and video data sent to the server

[0891] Step 6:

[0892] The server analyzes the transmitted audio and video data in real time, uses an emotion engine to recognize the applicant's emotional state and reflect it in the evaluation, and generates feedback in real time and sends it to the user's device.

[0893] Input: Audio and video data received in real time

[0894] Data Computing: Analysis of Audio-Visual Data and Recognition of Emotional States

[0895] Output: Feedback sent to the user's device

[0896] Step 7:

[0897] The server combines the document screening evaluation results with the interview results to make a final evaluation. It weights each evaluation point and determines an overall evaluation. It generates a recommended list of candidates for employment and sends it to the user's terminal. The user then reviews the final candidates and makes a final decision.

[0898] Input: Document review evaluation results and interview results

[0899] Data calculation: Weighting of each evaluation point and determining the overall evaluation

[0900] Output: A list of recommended job candidates sent to the user's device

[0901] (Application example 2)

[0902] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0903] Conventional recruitment interview systems have difficulty effectively managing applicant document screening and interviews, particularly due to a lack of real-time feedback and recognition of emotional states. This prevents interviewers from fully understanding the true skills and emotional states of applicants, resulting in an inefficient hiring process and potentially unfair decisions. The present invention aims to solve these problems.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for using the generated evaluation results to generate a set of interview questions, means for analyzing digital data collected during interviews in real time and providing feedback, means for analyzing collected audio and video data with an emotion recognition engine to recognize the applicant's emotional state in real time, means for allowing the interviewer to visually confirm the emotional state, and means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment. This makes it possible to grasp the applicant's skills and emotional state in real time and realize a fair and efficient hiring process.

[0905] "Applicant document data" refers to digital data that includes information such as resumes and curriculum vitae submitted by applicants.

[0906] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to evaluate an applicant's skills, experience, and educational background.

[0907] An "interview question set" is a list of questions that is individually created based on the evaluation results analyzed by the generative AI model and is used by interviewers with applicants.

[0908] The "emotion recognition engine" is a system that analyzes collected audio and video data and can recognize the emotional state of applicants in real time.

[0909] "Feedback" refers to assessments and advice provided in real time during the interview, which allows the interviewer to properly assess the candidate's skills and emotional state.

[0910] The "final evaluation" is a comprehensive evaluation that integrates the results of the document review and interviews, and serves as the basis for selecting candidates for employment.

[0911] An embodiment of the present invention is a system for collecting, analyzing, and evaluating document data from applicants, providing real-time feedback through interviews, and ultimately supporting hiring decisions. This system is composed of multiple elements, including a server, a user terminal, a generative AI model, and an emotion recognition engine.

[0912] System Configuration

[0913] 1. Server:

[0914] The server acts as the central control unit and performs all data processing and analysis.

[0915] The server collects applicants' document data and analyzes it using a generative AI model.

[0916] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[0917] The server analyzes the digital data collected during the interview in real time and provides feedback.

[0918] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[0919] 2. User Device:

[0920] A user terminal is a device used by a recruiter or interviewer.

[0921] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[0922] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[0923] 3. Generative AI Model:

[0924] The generative AI model is implemented on the server and is used to analyze applicant document data.

[0925] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[0926] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[0927] 4. Emotion Recognition Engine:

[0928] The emotion recognition engine analyzes audio and video data collected during the interview and recognizes the applicant's emotional state in real time.

[0929] The emotion recognition engine generates and provides feedback to the user terminal based on the recognized emotional state.

[0930] The emotion recognition engine has the function of visually displaying the emotional state.

[0931] Program processing explanation

[0932] 1. Data entry and storage:

[0933] Using a user terminal, a recruiter inputs job information into a server and stores it in a database.

[0934] 2. Document data collection and analysis:

[0935] Applicants submit their resumes and work histories via the application, which are then stored on the server.

[0936] The server uses a generative AI model to analyze these documents and generate a rating.

[0937] The results of this evaluation are used to generate a set of interview questions.

[0938] 3. Generate a set of interview questions:

[0939] The server uses a generative AI model to automatically generate a set of interview questions that are individually tailored.

[0940] This set of questions is sent to the recruiter's user terminal.

[0941] 4. Real-time analysis and feedback:

[0942] During the interview, the user's terminal transmits audio and video data to the server, which analyzes it in real time.

[0943] The server generates feedback based on this data and provides it to the user terminal.

[0944] 5. Emotion recognition:

[0945] The emotion recognition engine analyzes audio and video data to recognize the applicant's emotional state in real time.

[0946] The recognized emotional state is reflected in the evaluation and can be visually confirmed on the user's terminal.

[0947] 6. Final evaluation and recommendation:

[0948] The server will integrate the document review and interview results and make a final evaluation.

[0949] Based on the results of the final evaluation, we will recommend candidates for employment.

[0950] Specific examples

[0951] As an example, let's look at a use case in a store looking to hire a "software engineer." The store's recruiter would input the following prompts into the generative AI model on their smartphone:

[0952] Example prompt sentence:

[0953] "Analyze resumes, assess applicants' skills, years of experience, and educational background, and generate a set of interview questions, such as: 1. What role did you play in your most recent project? 2. Tell me about how you collaborate with your team. 3. Tell me more about the technology stack you used."

[0954] Based on this prompt, a generative AI model analyzes the applicant's documents and generates a set of interview questions that are individually tailored to the individual. The actual interview then takes place, and the server analyzes the data in real time, provides feedback, and visualizes the applicant's emotional state using an emotion recognition engine. Finally, the server integrates all the data, makes a final evaluation, and recommends suitable candidates for employment.

[0955] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0956] Step 1:

[0957] The user uses their own device to enter job information (job type, required skills, years of experience, etc.) into the corresponding application and sends it to the server. This data is stored in a database on the server. The input data includes details such as company name, position, required skills, etc. This allows the server to hold the job information necessary for subsequent processing.

[0958] Step 2:

[0959] Applicants upload their resumes, job history documents, and other documents to the server via the application. This data is stored in a database and used for further analysis. Specifically, documents in PDF and Word file formats are considered. This allows the server to collect detailed information about each applicant.

[0960] Step 3:

[0961] The server inputs the collected applicant document data into a generative AI model for analysis. The analysis uses natural language processing technology to extract the applicant's skills, experience, educational background, etc., and generates an evaluation result. This evaluation result includes details such as skill level, years of experience, and expertise. This allows for an objective evaluation of the applicant's suitability.

[0962] Step 4:

[0963] The server uses a generative AI model to automatically generate a set of interview questions based on the analyzed evaluation results. By inputting specific prompts, customized interview questions are generated for each applicant. For example, specific questions based on the applicant's technology stack or project experience can be included. This allows interviewers to prepare effective and relevant questions.

[0964] Step 5:

[0965] A set of interview questions is sent to the user's device, and the interviewer conducts the interview based on this. During the interview, audio and video data is sent from the user's device to the server in real time, providing input data for the server to analyze the collected data in real time.

[0966] Step 6:

[0967] The server analyzes the collected audio and video data in real time and generates feedback. An emotion recognition engine is used in the analysis to recognize the candidate's emotional state. Based on this, the server generates feedback and sends it to the user's device. This feedback includes an evaluation of the candidate's communication skills and emotional responses.

[0968] Step 7:

[0969] The emotion recognition engine recognizes the applicant's emotional state in real time based on the collected data. The recognition results are displayed visually on the user's device so that the interviewer can check them in real time. This display includes the type of emotion (e.g., joy, surprise, nervousness, etc.) and its intensity, making it easier for the interviewer to understand the applicant's emotional state.

[0970] Step 8:

[0971] After the interview, the server integrates the document review and interview results to make a final evaluation. This integration includes the applicant's skills, experience, interview performance, and emotion recognition results. Based on the final evaluation results, the server generates a list of recommended candidates and sends it to the user's device. This ensures fair and efficient hiring decisions.

[0972] Step 9:

[0973] Based on the final evaluation results, the user reviews the candidates and makes the final hiring decision. The final decision is sent from the user's device to the server and saved in the database. This records the entire hiring process and saves data that can be used for future reference and improvement.

[0974] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0975] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0976] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0977] [Third embodiment]

[0978] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0979] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0980] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0981] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0982] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0983] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0984] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0985] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0986] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0987] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0988] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0989] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0990] This system consists of multiple elements, including a server, a user terminal, and a generative AI model. Each element and its operation are explained in detail below.

[0991] System Configuration

[0992] 1. Server

[0993] The server acts as the central control unit and performs all data processing and analysis.

[0994] The server collects applicants' document data and analyzes it using a generative AI model.

[0995] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[0996] The server analyzes the data collected during the interview in real time and provides feedback.

[0997] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[0998] 2. User Device

[0999] The user terminal is a device used by HR personnel and interviewers.

[1000] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[1001] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[1002] 3. Generative AI Models

[1003] The generative AI model is implemented on the server and is used to analyze applicant document data.

[1004] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[1005] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[1006] Program processing

[1007] 1. User enters job information

[1008] Users input job information from their own devices and send it to the server, which then stores the received job information in a database.

[1009] 2. Collection and analysis of application documents

[1010] When an applicant submits a resume and work history, the server receives them and stores them in a database.

[1011] The server uses a generative AI model to analyze these documents and evaluate the applicant's skills, experience, and educational background.

[1012] The server generates an evaluation result based on the analysis result and transmits it to the user terminal.

[1013] 3. Generating a set of interview questions

[1014] The server generates an individually appropriate set of interview questions based on the analysis results.

[1015] The server transmits the generated interview question set to the user terminal.

[1016] 4. Conducting interviews and real-time evaluations

[1017] The user conducts an interview using the interview question set and transmits the voice data and text data collected during the interview to the server.

[1018] The server analyzes this data in real time and evaluates the candidate's communication skills and responses.

[1019] The server sends feedback to the user terminal in real time to support the evaluation procedure.

[1020] 5. Final evaluation and recommendation of job offers

[1021] The server will combine the results of the document review and interview and make a final evaluation.

[1022] The server selects job offer candidates based on the final evaluation results and transmits a recommendation list to the user terminal.

[1023] The user reviews the final candidates and makes a final decision.

[1024] Specific examples

[1025] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation results, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on these questions, and by sending data during the interview to the server, the server performs real-time analysis and provides feedback. Finally, the server integrates all the evaluation results, generates a recommended list of candidates for employment, and sends it to the user's device, thereby supporting fair hiring practices.

[1026] The processing flow will be explained below.

[1027] Step 1:

[1028] User enters job information

[1029] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own terminals.

[1030] The user terminal transmits the input information to the server.

[1031] Step 2:

[1032] The server collects the applications.

[1033] The server receives document data (resume, curriculum vitae, etc.) submitted by applicants and stores it in a database.

[1034] Step 3:

[1035] The server analyzes the document data

[1036] The server inputs the application documents into a generative AI model and uses natural language processing technology to analyze skills, experience, and educational background.

[1037] Step 4:

[1038] The server generates the evaluation results

[1039] The server generates a skill matching score, an aptitude score, and an overall evaluation from the analysis results.

[1040] The server transmits the generated evaluation results in a visualized report format to the terminal of the HR staff.

[1041] Step 5:

[1042] The server generates a set of interview questions

[1043] The server generates an individually appropriate set of interview questions based on the evaluation results.

[1044] The server transmits the generated interview question set to the user terminal.

[1045] Step 6:

[1046] User conducts interview

[1047] The user conducts an interview based on the question set sent from the server.

[1048] Step 7:

[1049] The user sends the interview data to the server

[1050] The user transmits the voice data and text data collected during the interview to the server in real time.

[1051] Step 8:

[1052] The server analyzes the interview data in real time

[1053] The server analyzes the received voice and text data and evaluates communication skills and responses.

[1054] Step 9:

[1055] The server provides feedback to the user device

[1056] The server generates real-time feedback during or after the interview and sends it to the HR representative's terminal.

[1057] Step 10:

[1058] The server performs the final evaluation

[1059] The server integrates the document review and interview results and calculates an overall evaluation score for each candidate.

[1060] Step 11:

[1061] The server selects job candidates and generates a recommendation list

[1062] The server selects job offer candidates based on the final evaluation scores.

[1063] The server transmits a recommended list of job offer candidates to the terminal of the HR person.

[1064] Step 12:

[1065] User makes final decision and sends offer letter

[1066] The user makes a final decision based on the recommended list of job candidates.

[1067] The user generates and sends a job offer notice to the determined job offer candidate.

[1068] Example 1

[1069] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1070] In traditional hiring processes, it is difficult to accurately evaluate an applicant's skills and experience, and preparing interview questions requires a great deal of effort. Furthermore, evaluations during interviews tend to be subjective, making it difficult to ensure fair hiring practices. Furthermore, integrating multiple data sets to make a final evaluation requires a great deal of time and expertise.

[1071] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1072] In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for saving the analysis results in a database and generating and displaying evaluation results in the form of a visual report, means for automatically generating a set of interview questions based on the analysis results, means for analyzing data collected during interviews in real time and providing feedback, and means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment. This enables accurate evaluation of applicants' skills and experience, generation of fair interview questions, real-time evaluation feedback, and comprehensive evaluation integrating multiple data.

[1073] "Applicant's document data" refers to information on documents submitted by applicants, such as resumes and job history documents.

[1074] "Means of collection" refers to the methods and functions for obtaining document data submitted by applicants and storing it within the system.

[1075] "Analyzing" refers to the detailed evaluation and analysis of collected document data using data processing techniques and generative AI models.

[1076] "Generative AI model" refers to an artificial intelligence algorithm or technology used to evaluate an applicant's skills, experience, educational background, etc.

[1077] "Evaluation results" refers to the analysis results of an applicant's skills, experience, and educational background analyzed using a generative AI model.

[1078] "Means for displaying in a visual report format" refers to a method or function for converting the evaluation results into a visually easy-to-understand format such as a graph or chart and displaying them to the user.

[1079] An "interview question set" refers to a list of questions to be used during an interview, created based on the applicant's evaluation results.

[1080] "Real-time" analysis refers to the immediate processing and evaluation of audio and text data collected during interviews.

[1081] "Feedback" refers to evaluations and suggestions provided in real time based on the analysis results.

[1082] "Means for recommending candidates for employment" refers to the method and function of selecting and recommending appropriate candidates for employment based on a comprehensive evaluation of document screening and interview results.

[1083] MODE FOR CARRYING OUT THE INVENTION

[1084] This invention is a system that uses a server, a user terminal, and a generative AI model to efficiently evaluate applicant document data and support interviews. This will be explained in detail below.

[1085] Hardware and Software Configuration

[1086] 1. Server:

[1087] The server acts as a central control unit, collecting applicant document data and analyzing the data using a generative AI model.

[1088] The server is equipped with a database that stores received job information, applicant document data, and analysis results.

[1089] Based on the analysis results, a set of interview questions is generated and sent to the user's terminal.

[1090] 2. User Device:

[1091] The user terminal is a device used by HR personnel and interviewers, and is used to input job information, check evaluation results, receive interview question sets, send interview results, and so on.

[1092] It provides tools to receive feedback from the server in real time and conduct interviews effectively.

[1093] 3. Generative AI Model:

[1094] The generative AI model is implemented on a server and uses natural language processing technology to analyze applicants' skills, experience, and educational background.

[1095] A set of interview questions is automatically generated based on the analysis results.

[1096] Specific processing of the program

[1097] Collection and analysis of application documents

[1098] Resumes and work histories received from applicants are uploaded to the server in PDF or Word format. The server then uses OCR (Optical Character Recognition) technology to convert these documents into text data and stores it in a database. The generative AI model then analyzes the applicant's skills, experience, and educational background. The analysis results are stored in the database in numerical and graphical format and displayed on the user's device.

[1099] Generating a set of interview questions

[1100] Based on the analysis results of the generative AI model, the server automatically generates a set of interview questions suited to each applicant. For example, personalized questions are created based on the applicant's technical skills and past experience. This question set is sent to the user's device and can be used to help prepare for the interview.

[1101] Real-time interview evaluation and feedback

[1102] The user conducts an interview with an applicant using a set of interview questions and records audio and text data. This data is sent to the server in real time and analyzed by the generative AI model. The analysis results are fed back to the user's device in real time, providing an evaluation of communication skills and the quality of responses, thereby supporting the progress of the interview.

[1103] Final evaluation and recommendation of job candidates

[1104] The server integrates the results of the applicant's document review and interview evaluation to make a final evaluation. Based on the final evaluation results, a recommended list of candidates for employment is generated and sent to the user's terminal. The user then decides on the final candidates for employment based on this recommendation list.

[1105] Specific examples

[1106] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation result, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on this, and by sending data during the interview to the server, the server performs real-time analysis and provides feedback. Finally, the server integrates all the evaluation results and generates a recommended list of candidates for employment, supporting fair hiring practices.

[1107] Prompt Sentence Examples

[1108] "Enter your software engineer job posting and we'll collect and analyze your applications."

[1109] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1110] Step 1:

[1111] The user enters job information. The user uses a dedicated web form to enter job information such as job description, required skills, years of experience, etc., and sends it to the server. The server stores the received job information in a database.

[1112] Input: Job information (job description, required skills, years of experience, etc.)

[1113] Output: Job listings stored in a database

[1114] Specific operation: The server receives the job information as text data and stores it in the database using an SQL query.

[1115] Step 2:

[1116] The user collects the applicant's document data and sends it to the server. The applicant's resume and work history are uploaded in PDF or Word format. The server receives these files and stores them in a database.

[1117] Input: Resume, work history (PDF or Word format)

[1118] Output: Document data stored in a database

[1119] Specific operation: The server receives the uploaded file and stores it in a database along with the file name and applicant information.

[1120] Step 3:

[1121] The server analyzes the collected document data with a generative AI model to evaluate the applicant's skills, experience, and educational background. It uses OCR technology to convert the data into text. The generative AI model then analyzes the data using natural language processing technology.

[1122] Input: Document data (converted to text format)

[1123] Output: Analysis results (evaluation of skills, experience, and educational background)

[1124] How it works: The server uses OCR technology to extract text data from image files, then inputs the text data into a generative AI model for analysis. The analysis results are stored in a database.

[1125] Step 4:

[1126] The server generates a set of interview questions based on the analysis results.The server generates interview questions personalized for each applicant based on the analysis results.

[1127] Input: Analysis results (evaluation of skills, experience, and educational background)

[1128] Output: A set of interview questions

[1129] Specific operation: The server uses the generative AI model to generate a list of questions based on the analysis results, and the generated questions are stored in a database and sent to the user's device.

[1130] Step 5:

[1131] The user conducts an interview using a set of interview questions. The collected voice and text data is sent to a server and analyzed in real time. The server then analyzes the data using a generative AI model and provides feedback.

[1132] Input: Audio and text data collected during the interview

[1133] Output: Real-time feedback

[1134] Specific operation: The user conducts an interview and sends voice and text data to the server, which then analyzes this data in real time using a generative AI model and sends feedback to the user's device.

[1135] Step 6:

[1136] The server integrates the document screening and interview results, performs a final evaluation, and recommends candidates for employment. Based on the results of the final evaluation, the server creates a list of candidates for employment and sends the recommendation list to the user's terminal.

[1137] Input: Document review results, interview results

[1138] Output: Recommended list of job offers

[1139] Specific operation: The server integrates the document screening results and interview results, performs a comprehensive evaluation using a generative AI model, generates a list of job-success candidates based on the final evaluation, and sends it to the user's device.

[1140] (Application example 1)

[1141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1142] Conventional hiring processes have problems such as requiring a great deal of time and effort for document review of applicants, preparation of interview questions, and evaluation during interviews. Furthermore, real-time feedback is difficult, leading to inconsistent evaluation criteria for applicants. Large retail chains, in particular, often require an efficient and consistent hiring process. Therefore, the present invention was designed to solve these problems.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1144] In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for generating evaluation results and displaying them in a visual report format, means for using the generated evaluation results to generate an interview question set, means for analyzing data collected during the interview in real time and providing feedback, means for integrating the document screening and interview results, making a final evaluation, and recommending a candidate for employment, and means for displaying the interview question set and receiving real-time feedback through smart glasses, thereby improving the efficiency and consistency of applicant document screening and interviews and enabling real-time evaluation and feedback.

[1145] "Applicant document data" refers to a collection of information such as resumes and job histories submitted by job applicants, which contain information about an individual's skills, experience, educational background, etc.

[1146] A "generative AI model" is a computer program based on artificial intelligence technology that uses natural language processing and machine learning to evaluate applicants' skills, experience, educational background, etc.

[1147] A "visual report format" is a method of presenting analytical results in a visual format such as graphs, tables, or charts.

[1148] An "interview question set" is a set of questions to be used during an interview, generated based on the applicant's evaluation results.

[1149] "Analyzing data in real time" means instantly processing the audio and text data collected during the interview and obtaining the analysis results.

[1150] "Means of providing feedback" refers to a function that provides evaluations and advice to interviewers in real time during the interview based on the analysis results.

[1151] The "means of recommending candidates for job offers" refers to a method of integrating the results of document screening and interviews, conducting a final evaluation, and selecting and notifying suitable applicants for job offers.

[1152] "Smart glasses" are wearable devices equipped with a display and voice recognition functions, and are glasses-type terminals that can display information and receive instructions.

[1153] "Real-time feedback" refers to the instantaneous provision of data analysis results during the interview, providing evaluation information that the interviewer can use as a reference on the spot.

[1154] This invention is a system that collects, analyzes, and evaluates applicants' document data, generates interview question sets, provides real-time evaluation, feedback, and final evaluation. In particular, it uses smart glasses to streamline the interview process and provide real-time feedback.

[1155] System Configuration

[1156] 1. Server

[1157] Hardware: High-performance server unit (e.g., Amazon Web Services EC2 instance)

[1158] Software: Generative AI models running on servers, database systems (e.g., MySQL), and speech analysis software (e.g., Google Speech-to-Text API)

[1159] Functions: Document data collection, analysis by generative AI model, visual display of evaluation results, generation of interview question sets, real-time analysis and feedback, final evaluation and recommendation of job offers

[1160] 2. User Device

[1161] Hardware: Smart glasses (e.g. Google Glass), PC or tablet

[1162] Software: Dedicated application installed on the smart glasses

[1163] Functions: Enter job information, view interview question sets, receive real-time feedback, submit interview results, and check final evaluation results

[1164] System Operation

[1165] 1. Enter job information

[1166] The user uses the smart glasses to input job information by voice or by selecting from a simple menu, and then sends the information to the server.

[1167] 2. Collection and analysis of application documents

[1168] The server receives document data such as resumes and work histories submitted by applicants and stores them in a database. The server then analyzes the documents using a generative AI model (e.g., a natural language processing model such as BERT or GPT-3) to evaluate the applicant's skills, experience, and educational background.

[1169] 3. Generating and displaying interview question sets

[1170] Based on the analysis results, the server generates a set of interview questions and displays them on the user's smart glasses. This set of questions is personalized based on the applicant's evaluation results.

[1171] 4. Real-time evaluation

[1172] The voice and text data collected during the interview is sent from the smart glasses to a server and analyzed in real time using a generative AI model and voice analysis software, with the analysis results provided as immediate feedback to the smart glasses.

[1173] 5. Final evaluation and recommendation of job offers

[1174] The server integrates the applicant's document screening and interview results to make a final evaluation. Based on the evaluation results, it selects candidates for employment and sends a recommendation list to the user's device.

[1175] Specific examples

[1176] For example, when an HR person at a retail chain is looking to recruit a new cashier, they use smart glasses to enter the job information into a server. The server receives and analyzes the applicant's document data and generates an evaluation result. Based on the generated evaluation result, a set of interview questions is created and displayed on the HR person's smart glasses during the interview. Data analysis and real-time feedback are provided during the interview, and finally, the server integrates all the data to make a final evaluation and recommend candidates for employment.

[1177] Prompt Sentence Examples

[1178] "You are using smart glasses to interview applicants. What features do you want from smart glasses?"

[1179] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1180] Step 1:

[1181] Users use the smart glasses to input job information by voice or menu selection, and then send it to the server. The input is collected as voice data and converted into text using speech recognition software such as the Google Speech-to-Text API. The converted job information is then sent to the server and stored in a database.

[1182] Input: Audio data

[1183] Output: Text data of job information

[1184] Step 2:

[1185] The server receives document data such as resumes and curriculum vitae submitted by applicants and stores them in a database. The server then analyzes these documents using a generative AI model (e.g., BERT or GPT-3) to evaluate the applicants' skills, experience, and educational background.

[1186] Input: Applicant's document data

[1187] Output: Analysis results (evaluation data)

[1188] Step 3:

[1189] The server generates a set of interview questions based on the analysis results. The generative AI model takes into account the applicant's evaluation results and creates an optimal set of questions. The set of questions is then displayed on the user's smart glasses.

[1190] Input: Analysis results (evaluation data)

[1191] Output: Interview question set

[1192] Step 4:

[1193] The user conducts an interview based on a set of interview questions displayed on the smart glasses. During the interview, the smart glasses collect the applicant's voice and text data and transmit it to a server in real time. The server then analyzes the collected data using voice analysis software and a generative AI model to evaluate the applicant's communication skills and answers.

[1194] Input: Voice data and text data during the interview

[1195] Output: Real-time analysis results

[1196] Step 5:

[1197] The server generates feedback during the interview based on the results of real-time analysis and immediately transmits the feedback to the user's smart glasses, which the user can use to progress and evaluate the interview.

[1198] Input: Real-time analysis results

[1199] Output: Real-time feedback

[1200] Step 6:

[1201] The server integrates the collected document data and interview results to make a final evaluation. Based on this data, it selects candidates for employment and generates a recommendation list. The final evaluation and recommendation list are sent to the user's smart glasses for viewing.

[1202] Input: Document data, interview results

[1203] Output: Final evaluation results, recommendation list of job offers

[1204] Specific examples

[1205] For example, when a retail chain's HR staff is recruiting cashiers, they use smart glasses to input job information, receive document data from applicants, and analyze it. Based on the generated question set, they conduct interviews and evaluate candidates while receiving real-time feedback. Finally, the server integrates all the evaluation data and recommends candidates for employment.

[1206] Prompt Sentence Examples

[1207] "You are using smart glasses to interview applicants. What features do you want from smart glasses?"

[1208] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1209] This system consists of multiple elements, including a server, a user device, a generative AI model, and an emotion engine. Below, we will explain each element and its operation in detail.

[1210] System Configuration

[1211] 1. Server

[1212] The server acts as the central control unit and performs all data processing and analysis.

[1213] The server collects applicants' document data and analyzes it using a generative AI model.

[1214] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[1215] The server analyzes the data collected during the interview in real time and provides feedback.

[1216] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[1217] 2. User Device

[1218] The user terminal is a device used by HR personnel and interviewers.

[1219] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[1220] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[1221] 3. Generative AI Models

[1222] The generative AI model is implemented on the server and is used to analyze applicant document data.

[1223] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[1224] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[1225] 4. Emotion Engine

[1226] The emotion engine analyzes audio and video data collected during the interview to recognize the user's emotional state in real time.

[1227] The emotion engine generates and provides feedback to the user terminal based on the recognized emotional state.

[1228] The emotion engine has the function of visually displaying the emotional state.

[1229] Program processing

[1230] 1. User enters job information

[1231] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own devices and send it to the server, which then stores the received recruitment information in a database.

[1232] 2. Collection and analysis of application documents

[1233] When an applicant submits a resume and work history, the server receives them and stores them in a database.

[1234] The server uses a generative AI model to analyze these documents and evaluate the applicant's skills, experience, and educational background.

[1235] The server generates an evaluation result based on the analysis result and transmits it to the user terminal.

[1236] 3. Generating a set of interview questions

[1237] The server generates an individually appropriate set of interview questions based on the analysis results.

[1238] The server transmits the generated interview question set to the user terminal.

[1239] 4. Conducting interviews and real-time evaluations

[1240] The user conducts an interview using the interview question set and transmits the audio and video data collected during the interview to the server.

[1241] The server analyzes this data in real time and evaluates the candidate's communication skills and responses.

[1242] 5. Emotion Recognition by Emotion Engine

[1243] The emotion engine analyzes audio and video data during the interview and recognizes the user's emotional state in real time.

[1244] The recognized emotional state is reflected in the evaluation and is provided to the user terminal as feedback.

[1245] The feedback provides a visual display of emotional state that can be seen by the interviewer in real time.

[1246] 6. Final evaluation and recommendation of job offers

[1247] The server will combine the results of the document review and interview and make a final evaluation.

[1248] The server selects job offer candidates based on the final evaluation results and transmits a recommendation list to the user terminal.

[1249] The user reviews the final candidates and makes a final decision.

[1250] Specific examples

[1251] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation results, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on this and sends the data from the interview to the server, which performs real-time analysis and provides feedback. The emotion engine analyzes the audio and video data during the interview to recognize the user's emotional state in real time and reflects the results in the evaluation. Finally, the server integrates all the evaluation results, generates a recommended list of candidates, and sends it to the user's device, thereby supporting fair hiring.

[1252] The processing flow will be explained below.

[1253] Step 1:

[1254] User enters job information

[1255] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own terminals.

[1256] The user sends the entered information to the server.

[1257] The server stores the received job information in a database.

[1258] Step 2:

[1259] The server collects the applications.

[1260] Applicants submit resumes and work history.

[1261] The server receives these document data and stores them in a database.

[1262] Step 3:

[1263] The server analyzes the document data

[1264] The server inputs the collected document data into a generative AI model.

[1265] The generative AI model uses natural language processing technology to analyze applicants' skills, experience, and educational background.

[1266] Step 4:

[1267] The server generates the evaluation results

[1268] The server generates a skill matching score, an aptitude score, and an overall evaluation based on the analysis results of the generative AI model.

[1269] The server transmits the generated evaluation results in the form of a report to the user terminal.

[1270] Step 5:

[1271] The server generates a set of interview questions

[1272] The server generates an individually tailored set of interview questions based on the evaluation results.

[1273] The server transmits the generated interview question set to the user terminal.

[1274] Step 6:

[1275] User conducts interview

[1276] The user conducts an interview based on the question set sent from the server.

[1277] The user collects audio and video data during the interview and transmits it to the server.

[1278] Step 7:

[1279] The server analyzes the interview data in real time

[1280] The server analyzes the collected audio and video data in real time.

[1281] The server uses a generative AI model and an emotion engine to evaluate the applicant's communication skills and emotional state.

[1282] Step 8:

[1283] The server provides feedback to the user device

[1284] The server generates evaluation results in real time and transmits them to the user terminal.

[1285] Users adjust the progress of the interview based on real-time feedback.

[1286] Step 9:

[1287] The server visually displays the emotional state

[1288] Secondarily, the emotion engine generates data that visually represents the recognized emotional state.

[1289] The server transmits visual representation data of the emotional state to the user terminal.

[1290] The user terminal visually displays the emotional state as the interview progresses.

[1291] Step 10:

[1292] The server performs the final evaluation

[1293] The server integrates the document review and interview results and calculates an overall evaluation score for each applicant.

[1294] The server transmits the final evaluation result to the user terminal.

[1295] Step 11:

[1296] The server selects job candidates and generates a recommendation list

[1297] The server selects job offer candidates based on the final evaluation scores.

[1298] The server generates a recommendation list of job offer candidates and transmits it to the user terminal.

[1299] Step 12:

[1300] User makes final decision and sends offer letter

[1301] The user makes the final decision based on the recommendation list.

[1302] The user generates and sends a job offer notice to the selected job offer candidate.

[1303] Example 2

[1304] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1305] In conventional hiring processes, document screening and interview evaluation of applicants are heavily dependent on human subjectivity, making it difficult to ensure fairness and accuracy. It is also difficult to properly assess an applicant's emotional state and communication skills during the interview, creating a need for methods to improve the quality of interview evaluations. Furthermore, the personalization of interview question sets is insufficient, making it impossible to ask questions tailored to the characteristics of each applicant.

[1306] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for generating evaluation results and displaying them in a visual report format, means for using the generated evaluation results to generate an interview question set, means for analyzing data collected during interviews in real time and providing feedback, means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment, and means for analyzing audio and video data during interviews using an emotion engine to recognize the applicant's emotional state and reflect it in the evaluation. This enables objective evaluation of the applicant's skills and experience and real-time analysis of the applicant's emotional state and communication skills during the interview, improving the fairness and accuracy of the hiring process.

[1307] "Applicant's document data" refers to document information such as resumes and curriculum vitae submitted by applicants.

[1308] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to analyze data and evaluate applicants' skills, experience, and educational background.

[1309] "Displayed in a visual report format" refers to a method of presenting analytical results as visual information such as graphs and charts.

[1310] An "interview question set" refers to a series of questions used during an interview, personalized based on the applicant's evaluation results.

[1311] "Emotion engine" refers to a system that analyzes audio and video data and recognizes emotional states in real time.

[1312] "Feedback" refers to information or advice generated based on data analyzed during the interview.

[1313] "User terminal" refers to a device such as a computer or smartphone operated by a user.

[1314] "Real-time analytics" refers to analytics methods in which data is processed immediately as it is collected.

[1315] The system for implementing this invention is composed of multiple elements, including a server, a user terminal, a generative AI model, and an emotion engine. Each element and its operation will be specifically described below.

[1316] 1. Server

[1317] The server acts as the central control unit of the system and performs all data processing and analysis. The server mainly uses the following software and hardware:

[1318] A database management system (e.g., MySQL, PostgreSQL) is used to store applicant document data and job information.

[1319] Generative AI models (e.g., TensorFlow, PyTorch) are used to analyze applicants' resumes and CVs to assess their skills, experience, and educational background.

[1320] Using real-time analytics software, audio and video data collected during interviews is analyzed to assess candidates' communication skills and emotional state.

[1321] 2. User Device

[1322] User terminals are devices used by HR personnel and interviewers and have the following functions:

[1323] It provides an input interface for job information and transmits the input information to the server.

[1324] Receives and visually displays generated assessment results and interview question sets.

[1325] It has a communication function for sending data collected during the interview to a server in real time.

[1326] 3. Generative AI Models

[1327] The generative AI model is implemented on a server and analyzes applicant document data using the following techniques:

[1328] Using natural language processing techniques (e.g., BERT, GPT-3), we extract important information from applicant documents and evaluate that information.

[1329] A deep learning algorithm automatically generates a set of interview questions based on the analysis results.

[1330] 4. Emotion Engine

[1331] The Emotion Engine is a system that analyzes audio and video data collected during interviews to recognize emotional states in real time. This engine has the following functions:

[1332] Analyze voice data using voice recognition technology (e.g., Kaldi, DeepSpeech).

[1333] Using video analysis technology (e.g., OpenCV, Dlib), facial expressions and gestures are analyzed from video data to recognize emotional states.

[1334] The recognized emotional state is provided as feedback to the user terminal in real time, and the emotional state is visually displayed.

[1335] Specific examples

[1336] If a company posts a job opening for a "software engineer," the user enters the job information on their own device and sends it to the server. When the applicant submits a resume and work history, the server receives it, analyzes it using a generative AI model, and generates an evaluation result for the applicant. Based on the evaluation results, the server generates a personalized set of interview questions and sends them to the user's device. The user conducts an interview based on this question set, and audio and video data from the interview is sent to the server in real time. Using an emotion engine, the server analyzes this data, recognizes the user's emotional state in real time, and reflects it in the evaluation. Finally, the server integrates all the evaluation results and generates a recommended list of candidates for employment, which is sent to the user's device.

[1337] Prompt Sentence Examples

[1338] "Use this system to input a job posting for a 'Software Engineer' and analyze applicants' resumes and CVs. It also generates a set of interview questions and assesses the candidate's communication skills and emotional state in real time during the interview."

[1339] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1340] Step 1:

[1341] Users input company recruitment information from their own devices and send it to the server. The input recruitment information includes job type, required skills, years of experience, etc. The server saves the received recruitment information in a database. Specifically, it checks the format of the recruitment information and saves it in the database appropriately.

[1342] Input: Job information such as job type, required skills, years of experience, etc.

[1343] Data processing: Check the format of input information

[1344] Output: Job listings stored in a database

[1345] Step 2:

[1346] When an applicant submits a resume or work history, the server receives it and stores it in a database. The server checks the file format and stores it in the database appropriately.

[1347] Input: Applicant's resume and work history

[1348] Data processing: Checking file format and saving to database

[1349] Output: Document data stored in a database

[1350] Step 3:

[1351] The server analyzes the application documents using a generative AI model. Specifically, it uses natural language processing technology to extract the applicant's skills, experience, and educational background, and calculates evaluation points. Based on the analysis results, it generates evaluation results and sends them to the user's device.

[1352] Input: Saved applicant resume and CV

[1353] Data Calculation: Information Extraction and Evaluation Point Calculation Using Natural Language Processing

[1354] Output: Evaluation results sent to the user's device

[1355] Step 4:

[1356] The server generates an optimal interview question set for each applicant based on the analysis results. The generative AI model automatically selects relevant questions and creates a question set. The created question set is then sent to the user's device.

[1357] Input: Parsed evaluation results

[1358] Data Computation: Automatic Question Generation

[1359] Output: A set of interview questions sent to the user's terminal

[1360] Step 5:

[1361] The user conducts an interview based on the interview question set using a user terminal. Audio and video data are collected during the interview. The terminal transmits this data to the server in real time.

[1362] Input: Audio and video data collected during the interview

[1363] Data processing: Real-time collection and transmission of audio and video data

[1364] Output: Audio and video data sent to the server

[1365] Step 6:

[1366] The server analyzes the transmitted audio and video data in real time, uses an emotion engine to recognize the applicant's emotional state and reflect it in the evaluation, and generates feedback in real time and sends it to the user's device.

[1367] Input: Audio and video data received in real time

[1368] Data Computing: Analysis of Audio-Visual Data and Recognition of Emotional States

[1369] Output: Feedback sent to the user's device

[1370] Step 7:

[1371] The server combines the document screening evaluation results with the interview results to make a final evaluation. It weights each evaluation point and determines an overall evaluation. It generates a recommended list of candidates for employment and sends it to the user's terminal. The user then reviews the final candidates and makes a final decision.

[1372] Input: Document review evaluation results and interview results

[1373] Data calculation: Weighting of each evaluation point and determining the overall evaluation

[1374] Output: A list of recommended job candidates sent to the user's device

[1375] (Application example 2)

[1376] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1377] Conventional recruitment interview systems have difficulty effectively managing applicant document screening and interviews, particularly due to a lack of real-time feedback and recognition of emotional states. This prevents interviewers from fully understanding the true skills and emotional states of applicants, resulting in an inefficient hiring process and potentially unfair decisions. The present invention aims to solve these problems.

[1378] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for using the generated evaluation results to generate a set of interview questions, means for analyzing digital data collected during interviews in real time and providing feedback, means for analyzing collected audio and video data with an emotion recognition engine to recognize the applicant's emotional state in real time, means for allowing the interviewer to visually confirm the emotional state, and means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment. This makes it possible to grasp the applicant's skills and emotional state in real time and realize a fair and efficient hiring process.

[1379] "Applicant document data" refers to digital data that includes information such as resumes and curriculum vitae submitted by applicants.

[1380] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to evaluate an applicant's skills, experience, and educational background.

[1381] An "interview question set" is a list of questions that is individually created based on the evaluation results analyzed by the generative AI model and is used by interviewers with applicants.

[1382] The "emotion recognition engine" is a system that analyzes collected audio and video data and can recognize the emotional state of applicants in real time.

[1383] "Feedback" refers to assessments and advice provided in real time during the interview, which allows the interviewer to properly assess the candidate's skills and emotional state.

[1384] The "final evaluation" is a comprehensive evaluation that integrates the results of the document review and interviews, and serves as the basis for selecting candidates for employment.

[1385] An embodiment of the present invention is a system for collecting, analyzing, and evaluating document data from applicants, providing real-time feedback through interviews, and ultimately supporting hiring decisions. This system is composed of multiple elements, including a server, a user terminal, a generative AI model, and an emotion recognition engine.

[1386] System Configuration

[1387] 1. Server:

[1388] The server acts as the central control unit and performs all data processing and analysis.

[1389] The server collects applicants' document data and analyzes it using a generative AI model.

[1390] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[1391] The server analyzes the digital data collected during the interview in real time and provides feedback.

[1392] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[1393] 2. User Device:

[1394] A user terminal is a device used by a recruiter or interviewer.

[1395] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[1396] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[1397] 3. Generative AI Model:

[1398] The generative AI model is implemented on the server and is used to analyze applicant document data.

[1399] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[1400] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[1401] 4. Emotion Recognition Engine:

[1402] The emotion recognition engine analyzes audio and video data collected during the interview and recognizes the applicant's emotional state in real time.

[1403] The emotion recognition engine generates and provides feedback to the user terminal based on the recognized emotional state.

[1404] The emotion recognition engine has the function of visually displaying the emotional state.

[1405] Program processing explanation

[1406] 1. Data entry and storage:

[1407] Using a user terminal, a recruiter inputs job information into a server and stores it in a database.

[1408] 2. Document data collection and analysis:

[1409] Applicants submit their resumes and work histories via the application, which are then stored on the server.

[1410] The server uses a generative AI model to analyze these documents and generate a rating.

[1411] The results of this evaluation are used to generate a set of interview questions.

[1412] 3. Generate a set of interview questions:

[1413] The server uses a generative AI model to automatically generate a set of interview questions that are individually tailored.

[1414] This set of questions is sent to the recruiter's user terminal.

[1415] 4. Real-time analysis and feedback:

[1416] During the interview, the user's terminal transmits audio and video data to the server, which analyzes it in real time.

[1417] The server generates feedback based on this data and provides it to the user terminal.

[1418] 5. Emotion recognition:

[1419] The emotion recognition engine analyzes audio and video data to recognize the applicant's emotional state in real time.

[1420] The recognized emotional state is reflected in the evaluation and can be visually confirmed on the user's terminal.

[1421] 6. Final evaluation and recommendation:

[1422] The server will integrate the document review and interview results and make a final evaluation.

[1423] Based on the results of the final evaluation, we will recommend candidates for employment.

[1424] Specific examples

[1425] As an example, let's look at a use case in a store looking to hire a "software engineer." The store's recruiter would input the following prompts into the generative AI model on their smartphone:

[1426] Example prompt sentence:

[1427] "Analyze resumes, assess applicants' skills, years of experience, and educational background, and generate a set of interview questions, such as: 1. What role did you play in your most recent project? 2. Tell me about how you collaborate with your team. 3. Tell me more about the technology stack you used."

[1428] Based on this prompt, a generative AI model analyzes the applicant's documents and generates a set of interview questions that are individually tailored to the individual. The actual interview then takes place, and the server analyzes the data in real time, provides feedback, and visualizes the applicant's emotional state using an emotion recognition engine. Finally, the server integrates all the data, makes a final evaluation, and recommends suitable candidates for employment.

[1429] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1430] Step 1:

[1431] The user uses their own device to enter job information (job type, required skills, years of experience, etc.) into the corresponding application and sends it to the server. This data is stored in a database on the server. The input data includes details such as company name, position, required skills, etc. This allows the server to hold the job information necessary for subsequent processing.

[1432] Step 2:

[1433] Applicants upload their resumes, job history documents, and other documents to the server via the application. This data is stored in a database and used for further analysis. Specifically, documents in PDF and Word file formats are considered. This allows the server to collect detailed information about each applicant.

[1434] Step 3:

[1435] The server inputs the collected applicant document data into a generative AI model for analysis. The analysis uses natural language processing technology to extract the applicant's skills, experience, educational background, etc., and generates an evaluation result. This evaluation result includes details such as skill level, years of experience, and expertise. This allows for an objective evaluation of the applicant's suitability.

[1436] Step 4:

[1437] The server uses a generative AI model to automatically generate a set of interview questions based on the analyzed evaluation results. By inputting specific prompts, customized interview questions are generated for each applicant. For example, specific questions based on the applicant's technology stack or project experience can be included. This allows interviewers to prepare effective and relevant questions.

[1438] Step 5:

[1439] A set of interview questions is sent to the user's device, and the interviewer conducts the interview based on this. During the interview, audio and video data is sent from the user's device to the server in real time, providing input data for the server to analyze the collected data in real time.

[1440] Step 6:

[1441] The server analyzes the collected audio and video data in real time and generates feedback. An emotion recognition engine is used in the analysis to recognize the candidate's emotional state. Based on this, the server generates feedback and sends it to the user's device. This feedback includes an evaluation of the candidate's communication skills and emotional responses.

[1442] Step 7:

[1443] The emotion recognition engine recognizes the applicant's emotional state in real time based on the collected data. The recognition results are displayed visually on the user's device so that the interviewer can check them in real time. This display includes the type of emotion (e.g., joy, surprise, nervousness, etc.) and its intensity, making it easier for the interviewer to understand the applicant's emotional state.

[1444] Step 8:

[1445] After the interview, the server integrates the document review and interview results to make a final evaluation. This integration includes the applicant's skills, experience, interview performance, and emotion recognition results. Based on the final evaluation results, the server generates a list of recommended candidates and sends it to the user's device. This ensures fair and efficient hiring decisions.

[1446] Step 9:

[1447] Based on the final evaluation results, the user reviews the candidates and makes the final hiring decision. The final decision is sent from the user's device to the server and saved in the database. This records the entire hiring process and saves data that can be used for future reference and improvement.

[1448] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1449] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1450] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1451] [Fourth embodiment]

[1452] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1453] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1454] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1455] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1456] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1457] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1458] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1459] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1460] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1461] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1462] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1463] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1464] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1465] This system consists of multiple elements, including a server, a user terminal, and a generative AI model. Each element and its operation are explained in detail below.

[1466] System Configuration

[1467] 1. Server

[1468] The server acts as the central control unit and performs all data processing and analysis.

[1469] The server collects applicants' document data and analyzes it using a generative AI model.

[1470] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[1471] The server analyzes the data collected during the interview in real time and provides feedback.

[1472] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[1473] 2. User Device

[1474] The user terminal is a device used by HR personnel and interviewers.

[1475] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[1476] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[1477] 3. Generative AI Models

[1478] The generative AI model is implemented on the server and is used to analyze applicant document data.

[1479] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[1480] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[1481] Program processing

[1482] 1. User enters job information

[1483] Users input job information from their own devices and send it to the server, which then stores the received job information in a database.

[1484] 2. Collection and analysis of application documents

[1485] When an applicant submits a resume and work history, the server receives them and stores them in a database.

[1486] The server uses a generative AI model to analyze these documents and evaluate the applicant's skills, experience, and educational background.

[1487] The server generates an evaluation result based on the analysis result and transmits it to the user terminal.

[1488] 3. Generating a set of interview questions

[1489] The server generates an individually appropriate set of interview questions based on the analysis results.

[1490] The server transmits the generated interview question set to the user terminal.

[1491] 4. Conducting interviews and real-time evaluations

[1492] The user conducts an interview using the interview question set and transmits the voice data and text data collected during the interview to the server.

[1493] The server analyzes this data in real time and evaluates the candidate's communication skills and responses.

[1494] The server sends feedback to the user terminal in real time to support the evaluation procedure.

[1495] 5. Final evaluation and recommendation of job offers

[1496] The server will combine the results of the document review and interview and make a final evaluation.

[1497] The server selects job offer candidates based on the final evaluation results and transmits a recommendation list to the user terminal.

[1498] The user reviews the final candidates and makes a final decision.

[1499] Specific examples

[1500] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation results, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on these questions, and by sending data during the interview to the server, the server performs real-time analysis and provides feedback. Finally, the server integrates all the evaluation results, generates a recommended list of candidates for employment, and sends it to the user's device, thereby supporting fair hiring practices.

[1501] The processing flow will be explained below.

[1502] Step 1:

[1503] User enters job information

[1504] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own terminals.

[1505] The user terminal transmits the input information to the server.

[1506] Step 2:

[1507] The server collects the applications.

[1508] The server receives document data (resume, curriculum vitae, etc.) submitted by applicants and stores it in a database.

[1509] Step 3:

[1510] The server analyzes the document data

[1511] The server inputs the application documents into a generative AI model and uses natural language processing technology to analyze skills, experience, and educational background.

[1512] Step 4:

[1513] The server generates the evaluation results

[1514] The server generates a skill matching score, an aptitude score, and an overall evaluation from the analysis results.

[1515] The server transmits the generated evaluation results in a visualized report format to the terminal of the HR staff.

[1516] Step 5:

[1517] The server generates a set of interview questions

[1518] The server generates an individually appropriate set of interview questions based on the evaluation results.

[1519] The server transmits the generated interview question set to the user terminal.

[1520] Step 6:

[1521] User conducts interview

[1522] The user conducts an interview based on the question set sent from the server.

[1523] Step 7:

[1524] The user sends the interview data to the server

[1525] The user transmits the voice data and text data collected during the interview to the server in real time.

[1526] Step 8:

[1527] The server analyzes the interview data in real time

[1528] The server analyzes the received voice and text data and evaluates communication skills and responses.

[1529] Step 9:

[1530] The server provides feedback to the user device

[1531] The server generates real-time feedback during or after the interview and sends it to the HR representative's terminal.

[1532] Step 10:

[1533] The server performs the final evaluation

[1534] The server integrates the document review and interview results and calculates an overall evaluation score for each candidate.

[1535] Step 11:

[1536] The server selects job candidates and generates a recommendation list

[1537] The server selects job offer candidates based on the final evaluation scores.

[1538] The server transmits a recommended list of job offer candidates to the terminal of the HR person.

[1539] Step 12:

[1540] User makes final decision and sends offer letter

[1541] The user makes a final decision based on the recommended list of job candidates.

[1542] The user generates and sends a job offer notice to the determined job offer candidate.

[1543] Example 1

[1544] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1545] In traditional hiring processes, it is difficult to accurately evaluate an applicant's skills and experience, and preparing interview questions requires a great deal of effort. Furthermore, evaluations during interviews tend to be subjective, making it difficult to ensure fair hiring practices. Furthermore, integrating multiple data sets to make a final evaluation requires a great deal of time and expertise.

[1546] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1547] In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for saving the analysis results in a database and generating and displaying evaluation results in the form of a visual report, means for automatically generating a set of interview questions based on the analysis results, means for analyzing data collected during interviews in real time and providing feedback, and means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment. This enables accurate evaluation of applicants' skills and experience, generation of fair interview questions, real-time evaluation feedback, and comprehensive evaluation integrating multiple data.

[1548] "Applicant's document data" refers to information on documents submitted by applicants, such as resumes and job history documents.

[1549] "Means of collection" refers to the methods and functions for obtaining document data submitted by applicants and storing it within the system.

[1550] "Analyzing" refers to the detailed evaluation and analysis of collected document data using data processing techniques and generative AI models.

[1551] "Generative AI model" refers to an artificial intelligence algorithm or technology used to evaluate an applicant's skills, experience, educational background, etc.

[1552] "Evaluation results" refers to the analysis results of an applicant's skills, experience, and educational background analyzed using a generative AI model.

[1553] "Means for displaying in a visual report format" refers to a method or function for converting the evaluation results into a visually easy-to-understand format such as a graph or chart and displaying them to the user.

[1554] An "interview question set" refers to a list of questions to be used during an interview, created based on the applicant's evaluation results.

[1555] "Real-time" analysis refers to the immediate processing and evaluation of audio and text data collected during interviews.

[1556] "Feedback" refers to evaluations and suggestions provided in real time based on the analysis results.

[1557] "Means for recommending candidates for employment" refers to the method and function of selecting and recommending appropriate candidates for employment based on a comprehensive evaluation of document screening and interview results.

[1558] MODE FOR CARRYING OUT THE INVENTION

[1559] This invention is a system that uses a server, a user terminal, and a generative AI model to efficiently evaluate applicant document data and support interviews. This will be explained in detail below.

[1560] Hardware and Software Configuration

[1561] 1. Server:

[1562] The server acts as a central control unit, collecting applicant document data and analyzing the data using a generative AI model.

[1563] The server is equipped with a database that stores received job information, applicant document data, and analysis results.

[1564] Based on the analysis results, a set of interview questions is generated and sent to the user's terminal.

[1565] 2. User Device:

[1566] The user terminal is a device used by HR personnel and interviewers, and is used to input job information, check evaluation results, receive interview question sets, send interview results, and so on.

[1567] It provides tools to receive feedback from the server in real time and conduct interviews effectively.

[1568] 3. Generative AI Model:

[1569] The generative AI model is implemented on a server and uses natural language processing technology to analyze applicants' skills, experience, and educational background.

[1570] A set of interview questions is automatically generated based on the analysis results.

[1571] Specific processing of the program

[1572] Collection and analysis of application documents

[1573] Resumes and work histories received from applicants are uploaded to the server in PDF or Word format. The server then uses OCR (Optical Character Recognition) technology to convert these documents into text data and stores it in a database. The generative AI model then analyzes the applicant's skills, experience, and educational background. The analysis results are stored in the database in numerical and graphical format and displayed on the user's device.

[1574] Generating a set of interview questions

[1575] Based on the analysis results of the generative AI model, the server automatically generates a set of interview questions suited to each applicant. For example, personalized questions are created based on the applicant's technical skills and past experience. This question set is sent to the user's device and can be used to help prepare for the interview.

[1576] Real-time interview evaluation and feedback

[1577] The user conducts an interview with an applicant using a set of interview questions and records audio and text data. This data is sent to the server in real time and analyzed by the generative AI model. The analysis results are fed back to the user's device in real time, providing an evaluation of communication skills and the quality of responses, thereby supporting the progress of the interview.

[1578] Final evaluation and recommendation of job candidates

[1579] The server integrates the results of the applicant's document review and interview evaluation to make a final evaluation. Based on the final evaluation results, a recommended list of candidates for employment is generated and sent to the user's terminal. The user then decides on the final candidates for employment based on this recommendation list.

[1580] Specific examples

[1581] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation result, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on this, and by sending data during the interview to the server, the server performs real-time analysis and provides feedback. Finally, the server integrates all the evaluation results and generates a recommended list of candidates for employment, supporting fair hiring practices.

[1582] Prompt Sentence Examples

[1583] "Enter your software engineer job posting and we'll collect and analyze your applications."

[1584] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1585] Step 1:

[1586] The user enters job information. The user uses a dedicated web form to enter job information such as job description, required skills, years of experience, etc., and sends it to the server. The server stores the received job information in a database.

[1587] Input: Job information (job description, required skills, years of experience, etc.)

[1588] Output: Job listings stored in a database

[1589] Specific operation: The server receives the job information as text data and stores it in the database using an SQL query.

[1590] Step 2:

[1591] The user collects the applicant's document data and sends it to the server. The applicant's resume and work history are uploaded in PDF or Word format. The server receives these files and stores them in a database.

[1592] Input: Resume, work history (PDF or Word format)

[1593] Output: Document data stored in a database

[1594] Specific operation: The server receives the uploaded file and stores it in a database along with the file name and applicant information.

[1595] Step 3:

[1596] The server analyzes the collected document data with a generative AI model to evaluate the applicant's skills, experience, and educational background. It uses OCR technology to convert the data into text. The generative AI model then analyzes the data using natural language processing technology.

[1597] Input: Document data (converted to text format)

[1598] Output: Analysis results (evaluation of skills, experience, and educational background)

[1599] How it works: The server uses OCR technology to extract text data from image files, then inputs the text data into a generative AI model for analysis. The analysis results are stored in a database.

[1600] Step 4:

[1601] The server generates a set of interview questions based on the analysis results.The server generates interview questions personalized for each applicant based on the analysis results.

[1602] Input: Analysis results (evaluation of skills, experience, and educational background)

[1603] Output: A set of interview questions

[1604] Specific operation: The server uses the generative AI model to generate a list of questions based on the analysis results, and the generated questions are stored in a database and sent to the user's device.

[1605] Step 5:

[1606] The user conducts an interview using a set of interview questions. The collected voice and text data is sent to a server and analyzed in real time. The server then analyzes the data using a generative AI model and provides feedback.

[1607] Input: Audio and text data collected during the interview

[1608] Output: Real-time feedback

[1609] Specific operation: The user conducts an interview and sends voice and text data to the server, which then analyzes this data in real time using a generative AI model and sends feedback to the user's device.

[1610] Step 6:

[1611] The server integrates the document screening and interview results, performs a final evaluation, and recommends candidates for employment. Based on the results of the final evaluation, the server creates a list of candidates for employment and sends the recommendation list to the user's terminal.

[1612] Input: Document review results, interview results

[1613] Output: Recommended list of job offers

[1614] Specific operation: The server integrates the document screening results and interview results, performs a comprehensive evaluation using a generative AI model, generates a list of job-success candidates based on the final evaluation, and sends it to the user's device.

[1615] (Application example 1)

[1616] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1617] Conventional hiring processes have problems such as requiring a great deal of time and effort for document review of applicants, preparation of interview questions, and evaluation during interviews. Furthermore, real-time feedback is difficult, leading to inconsistent evaluation criteria for applicants. Large retail chains, in particular, often require an efficient and consistent hiring process. Therefore, the present invention was designed to solve these problems.

[1618] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1619] In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for generating evaluation results and displaying them in a visual report format, means for using the generated evaluation results to generate an interview question set, means for analyzing data collected during the interview in real time and providing feedback, means for integrating the document screening and interview results, making a final evaluation, and recommending a candidate for employment, and means for displaying the interview question set and receiving real-time feedback through smart glasses, thereby improving the efficiency and consistency of applicant document screening and interviews and enabling real-time evaluation and feedback.

[1620] "Applicant document data" refers to a collection of information such as resumes and job histories submitted by job applicants, which contain information about an individual's skills, experience, educational background, etc.

[1621] A "generative AI model" is a computer program based on artificial intelligence technology that uses natural language processing and machine learning to evaluate applicants' skills, experience, educational background, etc.

[1622] A "visual report format" is a method of presenting analytical results in a visual format such as graphs, tables, or charts.

[1623] An "interview question set" is a set of questions to be used during an interview, generated based on the applicant's evaluation results.

[1624] "Analyzing data in real time" means instantly processing the audio and text data collected during the interview and obtaining the analysis results.

[1625] "Means of providing feedback" refers to a function that provides evaluations and advice to interviewers in real time during the interview based on the analysis results.

[1626] The "means of recommending candidates for job offers" refers to a method of integrating the results of document screening and interviews, conducting a final evaluation, and selecting and notifying suitable applicants for job offers.

[1627] "Smart glasses" are wearable devices equipped with a display and voice recognition functions, and are glasses-type terminals that can display information and receive instructions.

[1628] "Real-time feedback" refers to the instantaneous provision of data analysis results during the interview, providing evaluation information that the interviewer can use as a reference on the spot.

[1629] This invention is a system that collects, analyzes, and evaluates applicants' document data, generates interview question sets, provides real-time evaluation, feedback, and final evaluation. In particular, it uses smart glasses to streamline the interview process and provide real-time feedback.

[1630] System Configuration

[1631] 1. Server

[1632] Hardware: High-performance server unit (e.g., Amazon Web Services EC2 instance)

[1633] Software: Generative AI models running on servers, database systems (e.g., MySQL), and speech analysis software (e.g., Google Speech-to-Text API)

[1634] Functions: Document data collection, analysis by generative AI model, visual display of evaluation results, generation of interview question sets, real-time analysis and feedback, final evaluation and recommendation of job offers

[1635] 2. User Device

[1636] Hardware: Smart glasses (e.g. Google Glass), PC or tablet

[1637] Software: Dedicated application installed on the smart glasses

[1638] Functions: Enter job information, view interview question sets, receive real-time feedback, submit interview results, and check final evaluation results

[1639] System Operation

[1640] 1. Enter job information

[1641] The user uses the smart glasses to input job information by voice or by selecting from a simple menu, and then sends the information to the server.

[1642] 2. Collection and analysis of application documents

[1643] The server receives document data such as resumes and work histories submitted by applicants and stores them in a database. The server then analyzes the documents using a generative AI model (e.g., a natural language processing model such as BERT or GPT-3) to evaluate the applicant's skills, experience, and educational background.

[1644] 3. Generating and displaying interview question sets

[1645] Based on the analysis results, the server generates a set of interview questions and displays them on the user's smart glasses. This set of questions is personalized based on the applicant's evaluation results.

[1646] 4. Real-time evaluation

[1647] The voice and text data collected during the interview is sent from the smart glasses to a server and analyzed in real time using a generative AI model and voice analysis software, with the analysis results provided as immediate feedback to the smart glasses.

[1648] 5. Final evaluation and recommendation of job offers

[1649] The server integrates the applicant's document screening and interview results to make a final evaluation. Based on the evaluation results, it selects candidates for employment and sends a recommendation list to the user's device.

[1650] Specific examples

[1651] For example, when an HR person at a retail chain is looking to recruit a new cashier, they use smart glasses to enter the job information into a server. The server receives and analyzes the applicant's document data and generates an evaluation result. Based on the generated evaluation result, a set of interview questions is created and displayed on the HR person's smart glasses during the interview. Data analysis and real-time feedback are provided during the interview, and finally, the server integrates all the data to make a final evaluation and recommend candidates for employment.

[1652] Prompt Sentence Examples

[1653] "You are using smart glasses to interview applicants. What features do you want from smart glasses?"

[1654] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1655] Step 1:

[1656] Users use the smart glasses to input job information by voice or menu selection, and then send it to the server. The input is collected as voice data and converted into text using speech recognition software such as the Google Speech-to-Text API. The converted job information is then sent to the server and stored in a database.

[1657] Input: Audio data

[1658] Output: Text data of job information

[1659] Step 2:

[1660] The server receives document data such as resumes and curriculum vitae submitted by applicants and stores them in a database. The server then analyzes these documents using a generative AI model (e.g., BERT or GPT-3) to evaluate the applicants' skills, experience, and educational background.

[1661] Input: Applicant's document data

[1662] Output: Analysis results (evaluation data)

[1663] Step 3:

[1664] The server generates a set of interview questions based on the analysis results. The generative AI model takes into account the applicant's evaluation results and creates an optimal set of questions. The set of questions is then displayed on the user's smart glasses.

[1665] Input: Analysis results (evaluation data)

[1666] Output: Interview question set

[1667] Step 4:

[1668] The user conducts an interview based on a set of interview questions displayed on the smart glasses. During the interview, the smart glasses collect the applicant's voice and text data and transmit it to a server in real time. The server then analyzes the collected data using voice analysis software and a generative AI model to evaluate the applicant's communication skills and answers.

[1669] Input: Voice data and text data during the interview

[1670] Output: Real-time analysis results

[1671] Step 5:

[1672] The server generates feedback during the interview based on the results of real-time analysis and immediately transmits the feedback to the user's smart glasses, which the user can use to progress and evaluate the interview.

[1673] Input: Real-time analysis results

[1674] Output: Real-time feedback

[1675] Step 6:

[1676] The server integrates the collected document data and interview results to make a final evaluation. Based on this data, it selects candidates for employment and generates a recommendation list. The final evaluation and recommendation list are sent to the user's smart glasses for viewing.

[1677] Input: Document data, interview results

[1678] Output: Final evaluation results, recommendation list of job offers

[1679] Specific examples

[1680] For example, when a retail chain's HR staff is recruiting cashiers, they use smart glasses to input job information, receive document data from applicants, and analyze it. Based on the generated question set, they conduct interviews and evaluate candidates while receiving real-time feedback. Finally, the server integrates all the evaluation data and recommends candidates for employment.

[1681] Prompt Sentence Examples

[1682] "You are using smart glasses to interview applicants. What features do you want from smart glasses?"

[1683] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1684] This system consists of multiple elements, including a server, a user device, a generative AI model, and an emotion engine. Below, we will explain each element and its operation in detail.

[1685] System Configuration

[1686] 1. Server

[1687] The server acts as the central control unit and performs all data processing and analysis.

[1688] The server collects applicants' document data and analyzes it using a generative AI model.

[1689] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[1690] The server analyzes the data collected during the interview in real time and provides feedback.

[1691] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[1692] 2. User Device

[1693] The user terminal is a device used by HR personnel and interviewers.

[1694] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[1695] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[1696] 3. Generative AI Models

[1697] The generative AI model is implemented on the server and is used to analyze applicant document data.

[1698] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[1699] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[1700] 4. Emotion Engine

[1701] The emotion engine analyzes audio and video data collected during the interview to recognize the user's emotional state in real time.

[1702] The emotion engine generates and provides feedback to the user terminal based on the recognized emotional state.

[1703] The emotion engine has the function of visually displaying the emotional state.

[1704] Program processing

[1705] 1. User enters job information

[1706] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own devices and send it to the server, which then stores the received recruitment information in a database.

[1707] 2. Collection and analysis of application documents

[1708] When an applicant submits a resume and work history, the server receives them and stores them in a database.

[1709] The server uses a generative AI model to analyze these documents and evaluate the applicant's skills, experience, and educational background.

[1710] The server generates an evaluation result based on the analysis result and transmits it to the user terminal.

[1711] 3. Generating a set of interview questions

[1712] The server generates an individually appropriate set of interview questions based on the analysis results.

[1713] The server transmits the generated interview question set to the user terminal.

[1714] 4. Conducting interviews and real-time evaluations

[1715] The user conducts an interview using the interview question set and transmits the audio and video data collected during the interview to the server.

[1716] The server analyzes this data in real time and evaluates the candidate's communication skills and responses.

[1717] 5. Emotion Recognition by Emotion Engine

[1718] The emotion engine analyzes audio and video data during the interview and recognizes the user's emotional state in real time.

[1719] The recognized emotional state is reflected in the evaluation and is provided to the user terminal as feedback.

[1720] The feedback provides a visual display of emotional state that can be seen by the interviewer in real time.

[1721] 6. Final evaluation and recommendation of job offers

[1722] The server will combine the results of the document review and interview and make a final evaluation.

[1723] The server selects job offer candidates based on the final evaluation results and transmits a recommendation list to the user terminal.

[1724] The user reviews the final candidates and makes a final decision.

[1725] Specific examples

[1726] If a company posts a job opening for a "software engineer," the user enters the job information into the server and collects documents from applicants. The server analyzes the applicant's document data and generates an evaluation result. Based on the evaluation results, the server generates a set of interview questions and sends them to the user's device. The user then conducts an interview based on this and sends the data from the interview to the server, which performs real-time analysis and provides feedback. The emotion engine analyzes the audio and video data during the interview to recognize the user's emotional state in real time and reflects the results in the evaluation. Finally, the server integrates all the evaluation results, generates a recommended list of candidates, and sends it to the user's device, thereby supporting fair hiring.

[1727] The processing flow will be explained below.

[1728] Step 1:

[1729] User enters job information

[1730] Users input company recruitment information (job type, required skills, years of experience, etc.) from their own terminals.

[1731] The user sends the entered information to the server.

[1732] The server stores the received job information in a database.

[1733] Step 2:

[1734] The server collects the applications.

[1735] Applicants submit resumes and work history.

[1736] The server receives these document data and stores them in a database.

[1737] Step 3:

[1738] The server analyzes the document data

[1739] The server inputs the collected document data into a generative AI model.

[1740] The generative AI model uses natural language processing technology to analyze applicants' skills, experience, and educational background.

[1741] Step 4:

[1742] The server generates the evaluation results

[1743] The server generates a skill matching score, an aptitude score, and an overall evaluation based on the analysis results of the generative AI model.

[1744] The server transmits the generated evaluation results in the form of a report to the user terminal.

[1745] Step 5:

[1746] The server generates a set of interview questions

[1747] The server generates an individually tailored set of interview questions based on the evaluation results.

[1748] The server transmits the generated interview question set to the user terminal.

[1749] Step 6:

[1750] User conducts interview

[1751] The user conducts an interview based on the question set sent from the server.

[1752] The user collects audio and video data during the interview and transmits it to the server.

[1753] Step 7:

[1754] The server analyzes the interview data in real time

[1755] The server analyzes the collected audio and video data in real time.

[1756] The server uses a generative AI model and an emotion engine to evaluate the applicant's communication skills and emotional state.

[1757] Step 8:

[1758] The server provides feedback to the user device

[1759] The server generates evaluation results in real time and transmits them to the user terminal.

[1760] Users adjust the progress of the interview based on real-time feedback.

[1761] Step 9:

[1762] The server visually displays the emotional state

[1763] Secondarily, the emotion engine generates data that visually represents the recognized emotional state.

[1764] The server transmits visual representation data of the emotional state to the user terminal.

[1765] The user terminal visually displays the emotional state as the interview progresses.

[1766] Step 10:

[1767] The server performs the final evaluation

[1768] The server integrates the document review and interview results and calculates an overall evaluation score for each applicant.

[1769] The server transmits the final evaluation result to the user terminal.

[1770] Step 11:

[1771] The server selects job candidates and generates a recommendation list

[1772] The server selects job offer candidates based on the final evaluation scores.

[1773] The server generates a recommendation list of job offer candidates and transmits it to the user terminal.

[1774] Step 12:

[1775] User makes final decision and sends offer letter

[1776] The user makes the final decision based on the recommendation list.

[1777] The user generates and sends a job offer notice to the selected job offer candidate.

[1778] Example 2

[1779] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1780] In conventional hiring processes, document screening and interview evaluation of applicants are heavily dependent on human subjectivity, making it difficult to ensure fairness and accuracy. It is also difficult to properly assess an applicant's emotional state and communication skills during the interview, creating a need for methods to improve the quality of interview evaluations. Furthermore, the personalization of interview question sets is insufficient, making it impossible to ask questions tailored to the characteristics of each applicant.

[1781] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for generating evaluation results and displaying them in a visual report format, means for using the generated evaluation results to generate an interview question set, means for analyzing data collected during interviews in real time and providing feedback, means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment, and means for analyzing audio and video data during interviews using an emotion engine to recognize the applicant's emotional state and reflect it in the evaluation. This enables objective evaluation of the applicant's skills and experience and real-time analysis of the applicant's emotional state and communication skills during the interview, improving the fairness and accuracy of the hiring process.

[1782] "Applicant's document data" refers to document information such as resumes and curriculum vitae submitted by applicants.

[1783] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to analyze data and evaluate applicants' skills, experience, and educational background.

[1784] "Displayed in a visual report format" refers to a method of presenting analytical results as visual information such as graphs and charts.

[1785] An "interview question set" refers to a series of questions used during an interview, personalized based on the applicant's evaluation results.

[1786] "Emotion engine" refers to a system that analyzes audio and video data and recognizes emotional states in real time.

[1787] "Feedback" refers to information or advice generated based on data analyzed during the interview.

[1788] "User terminal" refers to a device such as a computer or smartphone operated by a user.

[1789] "Real-time analytics" refers to analytics methods in which data is processed immediately as it is collected.

[1790] The system for implementing this invention is composed of multiple elements, including a server, a user terminal, a generative AI model, and an emotion engine. Each element and its operation will be specifically described below.

[1791] 1. Server

[1792] The server acts as the central control unit of the system and performs all data processing and analysis. The server mainly uses the following software and hardware:

[1793] A database management system (e.g., MySQL, PostgreSQL) is used to store applicant document data and job information.

[1794] Generative AI models (e.g., TensorFlow, PyTorch) are used to analyze applicants' resumes and CVs to assess their skills, experience, and educational background.

[1795] Using real-time analytics software, audio and video data collected during interviews is analyzed to assess candidates' communication skills and emotional state.

[1796] 2. User Device

[1797] User terminals are devices used by HR personnel and interviewers and have the following functions:

[1798] It provides an input interface for job information and transmits the input information to the server.

[1799] Receives and visually displays generated assessment results and interview question sets.

[1800] It has a communication function for sending data collected during the interview to a server in real time.

[1801] 3. Generative AI Models

[1802] The generative AI model is implemented on a server and analyzes applicant document data using the following techniques:

[1803] Using natural language processing techniques (e.g., BERT, GPT-3), we extract important information from applicant documents and evaluate that information.

[1804] A deep learning algorithm automatically generates a set of interview questions based on the analysis results.

[1805] 4. Emotion Engine

[1806] The Emotion Engine is a system that analyzes audio and video data collected during interviews to recognize emotional states in real time. This engine has the following functions:

[1807] Analyze voice data using voice recognition technology (e.g., Kaldi, DeepSpeech).

[1808] Using video analysis technology (e.g., OpenCV, Dlib), facial expressions and gestures are analyzed from video data to recognize emotional states.

[1809] The recognized emotional state is provided as feedback to the user terminal in real time, and the emotional state is visually displayed.

[1810] Specific examples

[1811] If a company posts a job opening for a "software engineer," the user enters the job information on their own device and sends it to the server. When the applicant submits a resume and work history, the server receives it, analyzes it using a generative AI model, and generates an evaluation result for the applicant. Based on the evaluation results, the server generates a personalized set of interview questions and sends them to the user's device. The user conducts an interview based on this question set, and audio and video data from the interview is sent to the server in real time. Using an emotion engine, the server analyzes this data, recognizes the user's emotional state in real time, and reflects it in the evaluation. Finally, the server integrates all the evaluation results and generates a recommended list of candidates for employment, which is sent to the user's device.

[1812] Prompt Sentence Examples

[1813] "Use this system to input a job posting for a 'Software Engineer' and analyze applicants' resumes and CVs. It also generates a set of interview questions and assesses the candidate's communication skills and emotional state in real time during the interview."

[1814] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1815] Step 1:

[1816] Users input company recruitment information from their own devices and send it to the server. The input recruitment information includes job type, required skills, years of experience, etc. The server saves the received recruitment information in a database. Specifically, it checks the format of the recruitment information and saves it in the database appropriately.

[1817] Input: Job information such as job type, required skills, years of experience, etc.

[1818] Data processing: Check the format of input information

[1819] Output: Job listings stored in a database

[1820] Step 2:

[1821] When an applicant submits a resume or work history, the server receives it and stores it in a database. The server checks the file format and stores it in the database appropriately.

[1822] Input: Applicant's resume and work history

[1823] Data processing: Checking file format and saving to database

[1824] Output: Document data stored in a database

[1825] Step 3:

[1826] The server analyzes the application documents using a generative AI model. Specifically, it uses natural language processing technology to extract the applicant's skills, experience, and educational background, and calculates evaluation points. Based on the analysis results, it generates evaluation results and sends them to the user's device.

[1827] Input: Saved applicant resume and CV

[1828] Data Calculation: Information Extraction and Evaluation Point Calculation Using Natural Language Processing

[1829] Output: Evaluation results sent to the user's device

[1830] Step 4:

[1831] The server generates an optimal interview question set for each applicant based on the analysis results. The generative AI model automatically selects relevant questions and creates a question set. The created question set is then sent to the user's device.

[1832] Input: Parsed evaluation results

[1833] Data Computation: Automatic Question Generation

[1834] Output: A set of interview questions sent to the user's terminal

[1835] Step 5:

[1836] The user conducts an interview based on the interview question set using a user terminal. Audio and video data are collected during the interview. The terminal transmits this data to the server in real time.

[1837] Input: Audio and video data collected during the interview

[1838] Data processing: Real-time collection and transmission of audio and video data

[1839] Output: Audio and video data sent to the server

[1840] Step 6:

[1841] The server analyzes the transmitted audio and video data in real time, uses an emotion engine to recognize the applicant's emotional state and reflect it in the evaluation, and generates feedback in real time and sends it to the user's device.

[1842] Input: Audio and video data received in real time

[1843] Data Computing: Analysis of Audio-Visual Data and Recognition of Emotional States

[1844] Output: Feedback sent to the user's device

[1845] Step 7:

[1846] The server combines the document screening evaluation results with the interview results to make a final evaluation. It weights each evaluation point and determines an overall evaluation. It generates a recommended list of candidates for employment and sends it to the user's terminal. The user then reviews the final candidates and makes a final decision.

[1847] Input: Document review evaluation results and interview results

[1848] Data calculation: Weighting of each evaluation point and determining the overall evaluation

[1849] Output: A list of recommended job candidates sent to the user's device

[1850] (Application example 2)

[1851] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1852] Conventional recruitment interview systems have difficulty effectively managing applicant document screening and interviews, particularly due to a lack of real-time feedback and recognition of emotional states. This prevents interviewers from fully understanding the true skills and emotional states of applicants, resulting in an inefficient hiring process and potentially unfair decisions. The present invention aims to solve these problems.

[1853] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting document data of applicants, means for analyzing the collected document data and using a generative AI model to evaluate the applicant's skills, experience, and educational background, means for using the generated evaluation results to generate a set of interview questions, means for analyzing digital data collected during interviews in real time and providing feedback, means for analyzing collected audio and video data with an emotion recognition engine to recognize the applicant's emotional state in real time, means for allowing the interviewer to visually confirm the emotional state, and means for integrating the document screening and interview results, making a final evaluation, and recommending candidates for employment. This makes it possible to grasp the applicant's skills and emotional state in real time and realize a fair and efficient hiring process.

[1854] "Applicant document data" refers to digital data that includes information such as resumes and curriculum vitae submitted by applicants.

[1855] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to evaluate an applicant's skills, experience, and educational background.

[1856] An "interview question set" is a list of questions that is individually created based on the evaluation results analyzed by the generative AI model and is used by interviewers with applicants.

[1857] The "emotion recognition engine" is a system that analyzes collected audio and video data and can recognize the emotional state of applicants in real time.

[1858] "Feedback" refers to assessments and advice provided in real time during the interview, which allows the interviewer to properly assess the candidate's skills and emotional state.

[1859] The "final evaluation" is a comprehensive evaluation that integrates the results of the document review and interviews, and serves as the basis for selecting candidates for employment.

[1860] An embodiment of the present invention is a system for collecting, analyzing, and evaluating document data from applicants, providing real-time feedback through interviews, and ultimately supporting hiring decisions. This system is composed of multiple elements, including a server, a user terminal, a generative AI model, and an emotion recognition engine.

[1861] System Configuration

[1862] 1. Server:

[1863] The server acts as the central control unit and performs all data processing and analysis.

[1864] The server collects applicants' document data and analyzes it using a generative AI model.

[1865] The server generates an evaluation result based on the analysis result and generates a set of interview questions.

[1866] The server analyzes the digital data collected during the interview in real time and provides feedback.

[1867] The server integrates the document review and interview results, makes a final evaluation, and recommends candidates for job offer.

[1868] 2. User Device:

[1869] A user terminal is a device used by a recruiter or interviewer.

[1870] The user terminal is used to input job information, check evaluation results, receive interview question sets, and send interview results.

[1871] The user terminal receives feedback from the server in real time to support the evaluation procedure.

[1872] 3. Generative AI Model:

[1873] The generative AI model is implemented on the server and is used to analyze applicant document data.

[1874] The generative AI model uses natural language processing techniques to evaluate applicants' skills, experience, and educational background.

[1875] The generative AI model automatically generates a set of interview questions based on the evaluation results.

[1876] 4. Emotion Recognition Engine:

[1877] The emotion recognition engine analyzes audio and video data collected during the interview and recognizes the applicant's emotional state in real time.

[1878] The emotion recognition engine generates and provides feedback to the user terminal based on the recognized emotional state.

[1879] The emotion recognition engine has the function of visually displaying the emotional state.

[1880] Program processing explanation

[1881] 1. Data entry and storage:

[1882] Using a user terminal, a recruiter inputs job information into a server and stores it in a database.

[1883] 2. Document data collection and analysis:

[1884] Applicants submit their resumes and work histories via the application, which are then stored on the server.

[1885] The server uses a generative AI model to analyze these documents and generate a rating.

[1886] The results of this evaluation are used to generate a set of interview questions.

[1887] 3. Generate a set of interview questions:

[1888] The server uses a generative AI model to automatically generate a set of interview questions that are individually tailored.

[1889] This set of questions is sent to the recruiter's user terminal.

[1890] 4. Real-time analysis and feedback:

[1891] During the interview, the user's terminal transmits audio and video data to the server, which analyzes it in real time.

[1892] The server generates feedback based on this data and provides it to the user terminal.

[1893] 5. Emotion recognition:

[1894] The emotion recognition engine analyzes audio and video data to recognize the applicant's emotional state in real time.

[1895] The recognized emotional state is reflected in the evaluation and can be visually confirmed on the user's terminal.

[1896] 6. Final evaluation and recommendation:

[1897] The server will integrate the document review and interview results and make a final evaluation.

[1898] Based on the results of the final evaluation, we will recommend candidates for employment.

[1899] Specific examples

[1900] As an example, let's look at a use case in a store looking to hire a "software engineer." The store's recruiter would input the following prompts into the generative AI model on their smartphone:

[1901] Example prompt sentence:

[1902] "Analyze resumes, assess applicants' skills, years of experience, and educational background, and generate a set of interview questions, such as: 1. What role did you play in your most recent project? 2. Tell me about how you collaborate with your team. 3. Tell me more about the technology stack you used."

[1903] Based on this prompt, a generative AI model analyzes the applicant's documents and generates a set of interview questions that are individually tailored to the individual. The actual interview then takes place, and the server analyzes the data in real time, provides feedback, and visualizes the applicant's emotional state using an emotion recognition engine. Finally, the server integrates all the data, makes a final evaluation, and recommends suitable candidates for employment.

[1904] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1905] Step 1:

[1906] The user uses their own device to enter job information (job type, required skills, years of experience, etc.) into the corresponding application and sends it to the server. This data is stored in a database on the server. The input data includes details such as company name, position, required skills, etc. This allows the server to hold the job information necessary for subsequent processing.

[1907] Step 2:

[1908] Applicants upload their resumes, job history documents, and other documents to the server via the application. This data is stored in a database and used for further analysis. Specifically, documents in PDF and Word file formats are considered. This allows the server to collect detailed information about each applicant.

[1909] Step 3:

[1910] The server inputs the collected applicant document data into a generative AI model for analysis. The analysis uses natural language processing technology to extract the applicant's skills, experience, educational background, etc., and generates an evaluation result. This evaluation result includes details such as skill level, years of experience, and expertise. This allows for an objective evaluation of the applicant's suitability.

[1911] Step 4:

[1912] The server uses a generative AI model to automatically generate a set of interview questions based on the analyzed evaluation results. By inputting specific prompts, customized interview questions are generated for each applicant. For example, specific questions based on the applicant's technology stack or project experience can be included. This allows interviewers to prepare effective and relevant questions.

[1913] Step 5:

[1914] A set of interview questions is sent to the user's device, and the interviewer conducts the interview based on this. During the interview, audio and video data is sent from the user's device to the server in real time, providing input data for the server to analyze the collected data in real time.

[1915] Step 6:

[1916] The server analyzes the collected audio and video data in real time and generates feedback. An emotion recognition engine is used in the analysis to recognize the candidate's emotional state. Based on this, the server generates feedback and sends it to the user's device. This feedback includes an evaluation of the candidate's communication skills and emotional responses.

[1917] Step 7:

[1918] The emotion recognition engine recognizes the applicant's emotional state in real time based on the collected data. The recognition results are displayed visually on the user's device so that the interviewer can check them in real time. This display includes the type of emotion (e.g., joy, surprise, nervousness, etc.) and its intensity, making it easier for the interviewer to understand the applicant's emotional state.

[1919] Step 8:

[1920] After the interview, the server integrates the document review and interview results to make a final evaluation. This integration includes the applicant's skills, experience, interview performance, and emotion recognition results. Based on the final evaluation results, the server generates a list of recommended candidates and sends it to the user's device. This ensures fair and efficient hiring decisions.

[1921] Step 9:

[1922] Based on the final evaluation results, the user reviews the candidates and makes the final hiring decision. The final decision is sent from the user's device to the server and saved in the database. This records the entire hiring process and saves data that can be used for future reference and improvement.

[1923] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1924] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1925] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1926] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1927] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1928] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1929] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1930] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1931] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1932] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1933] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1934] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1935] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1936] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1937] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1938] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1939] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1940] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1941] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1942] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1943] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1944] The following is further disclosed regarding the above embodiment.

[1945] (Claim 1)

[1946] A means of collecting document data of applicants;

[1947] A means to use generative AI models that analyze collected document data and evaluate applicants' skills, experience, and educational background;

[1948] means for generating and displaying the results of the assessment in the form of a visual report;

[1949] A means for utilizing the generated evaluation results to generate a set of interview questions;

[1950] A means of analyzing data collected during interviews in real time and providing feedback;

[1951] A method for integrating the results of document screening and interviews, making a final evaluation, and recommending job offers.

[1952] A system including:

[1953] (Claim 2)

[1954] 2. The system of claim 1, wherein the set of interview questions is personalized based on the applicant's evaluation results.

[1955] (Claim 3)

[1956] 10. The system of claim 1, further comprising means for analyzing the voice data and text data collected during the interview to evaluate communication skills.

[1957] "Example 1"

[1958] (Claim 1)

[1959] A means of collecting document data of applicants;

[1960] A means to use generative AI models that analyze collected document data and evaluate applicants' skills, experience, and educational background;

[1961] means for storing the analysis results in a database and generating and displaying evaluation results in the form of a visual report;

[1962] A means for automatically generating a set of interview questions based on the analysis results;

[1963] A means of analyzing data collected during interviews in real time and providing feedback;

[1964] A method for integrating the results of document screening and interviews, making a final evaluation, and recommending job offers.

[1965] A system including:

[1966] (Claim 2)

[1967] 2. The system of claim 1, wherein the set of interview questions is personalized based on the applicant's evaluation results.

[1968] (Claim 3)

[1969] 10. The system of claim 1, further comprising means for analyzing the voice data and text data collected during the interview to evaluate communication skills.

[1970] "Application Example 1"

[1971] (Claim 1)

[1972] A means of collecting document data of applicants;

[1973] A means to use generative AI models that analyze collected document data and evaluate applicants' skills, experience, and educational background;

[1974] means for generating and displaying the results of the assessment in the form of a visual report;

[1975] A means for utilizing the generated evaluation results to generate a set of interview questions;

[1976] A means of analyzing data collected during interviews in real time and providing feedback;

[1977] A method for integrating the results of document screening and interviews, making a final evaluation, and recommending job offers.

[1978] a means for displaying a set of interview questions and receiving real-time feedback through the smart glasses;

[1979] A system including:

[1980] (Claim 2)

[1981] 10. The system of claim 1, wherein the set of interview questions is personalized based on the applicant's evaluation results and is displayed on the display of the smart glasses.

[1982] (Claim 3)

[1983] 10. The system of claim 1, utilizing smart glasses that include means for analyzing audio and text data collected during the interview to assess communication skills.

[1984] "Example 2: Combining Emotion Engines"

[1985] (Claim 1)

[1986] A means of collecting document data of applicants;

[1987] A means to use generative AI models that analyze collected document data and evaluate applicants' skills, experience, and educational background;

[1988] means for generating and displaying the results of the assessment in the form of a visual report;

[1989] A means for utilizing the generated evaluation results to generate a set of interview questions;

[1990] A means of analyzing data collected during interviews in real time and providing feedback;

[1991] A method for integrating the results of document screening and interviews, making a final evaluation, and recommending job offers.

[1992] a means for analyzing audio data and video data during an interview using an emotion engine, recognizing an emotional state, and reflecting the recognition in an evaluation;

[1993] A system including:

[1994] (Claim 2)

[1995] 2. The system of claim 1, wherein the set of interview questions is personalized based on the applicant's evaluation results.

[1996] (Claim 3)

[1997] 10. The system of claim 1, further comprising means for analyzing the voice data and text data collected during the interview to evaluate communication skills.

[1998] (Claim 4)

[1999] 2. The system according to claim 1, further comprising means for a user to input job information and transmit it to the server.

[2000] (Claim 5)

[2001] 2. The system according to claim 1, further comprising means for generating a set of interview questions based on the results of analyzing the applicant's documents and transmitting the set to the user terminal.

[2002] (Claim 6)

[2003] 10. The system of claim 1, further comprising means for analyzing data collected during the interview in real time and providing feedback from the server to the user terminal.

[2004] "Application example 2 when combining emotion engines"

[2005] (Claim 1)

[2006] A means of collecting document data of applicants;

[2007] A means to use generative AI models that analyze collected document data and evaluate applicants' skills, experience, and educational background;

[2008] means for generating and displaying the results of the assessment in the form of a visual report;

[2009] A means for utilizing the generated evaluation results to generate a set of interview questions;

[2010] A means of analyzing digital data collected during interviews in real time and providing feedback;

[2011] A means for analyzing the collected audio and video data with an emotion recognition engine to recognize the emotional state of the applicant in real time;

[2012] A means for the interviewer to visually confirm emotional state,

[2013] A method for integrating the results of document screening and interviews, making a final evaluation, and recommending job offers.

[2014] A system including:

[2015] (Claim 2)

[2016] 2. The system of claim 1, wherein the set of interview questions is personalized based on the applicant's evaluation results.

[2017] (Claim 3)

[2018] 10. The system of claim 1, further comprising means for analyzing audio and video data collected during the interview to assess communication skills and emotional state. [Explanation of symbols]

[2019] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting document data of applicants; A means to use generative AI models that analyze collected document data and evaluate applicants' skills, experience, and educational background; means for generating and displaying the results of the assessment in the form of a visual report; A means for utilizing the generated evaluation results to generate a set of interview questions; A means of analyzing data collected during interviews in real time and providing feedback; A method for integrating the results of document screening and interviews, making a final evaluation, and recommending job offers. A system including:

2. 2. The system of claim 1, wherein the set of interview questions is personalized based on the applicant's evaluation results.

3. 10. The system of claim 1, further comprising means for analyzing the voice data and text data collected during the interview to evaluate communication skills.

Images