System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting employment requirements of an enterprise; means for automatically generating a job recruiting slip based on the employment requirements; means for receiving a document submitted by a candidate; means for analyzing the document to extract skill and experience information; means for evaluating a matching degree of the candidate based on the skill and experience information; means for analyzing video and audio during an interview in real time and displaying an emotion analysis result; and means for scoring the matching degree of the candidate after the interview.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Acquiring talented personnel is a constant challenge for companies, and there is a serious shortage of talent in the recruitment of new graduates, mid-career hires, and contract employees. This increases the burden on on-site managers, making it difficult to process large volumes of application forms and resumes, especially in the early stages, and to properly evaluate candidates during interviews. Furthermore, these processes rely on the subjective judgment of managers, which creates a lack of objectivity. The purpose of this invention is to utilize AI to support recruitment activities, reduce the burden on managers, and enable them to make decisions from a more objective perspective. [Means for solving the problem]

[0005] The present invention is a system that includes a means for inputting a company's hiring requirements, a means for automatically generating a job posting based on the hiring requirements, a means for receiving documents submitted by candidates, a means for analyzing the documents to extract skill and experience information, a means for evaluating the candidate's match based on the skill and experience information, a means for analyzing video and audio during an interview in real time and displaying sentiment analysis results, a means for scoring the candidate's match after the interview is completed, and a means for comparatively evaluating multiple candidates based on the scoring results. The system further includes a means for performing sentiment analysis in real time during the interview and displaying candidate questions and concerns to the interviewer, thereby improving the quality of the interview. The system also includes a means for automatically generating a candidate suitability report based on the scoring results and providing it to the recruiter. This streamlines the entire hiring process and enables appropriate decisions based on objective data.

[0006] An "enterprise" is a corporation or sole proprietorship that engages in economic activities and is an organization that provides goods and services.

[0007] "Employment requirements" are the skills, experience, qualifications, characteristics, and other conditions that a company looks for when hiring new employees.

[0008] A "job posting" is a document provided by a company to job seekers that lists the types of jobs available, the job content, required skills, and compensation.

[0009] A "candidate" is a job seeker who wishes to be employed by a company and participates in the recruitment process.

[0010] A "document" is a document containing specific information written in the form of text or diagrams, and includes electronic data formats.

[0011] A resume is a document in which a candidate lists their work experience, job duties, and skills.

[0012] An "entry sheet" is a document that contains self-promotion and reasons for applying that candidates submit to companies, mainly for new graduate recruitment.

[0013] "Skills" are the knowledge, techniques, and abilities required to perform a specific task.

[0014] "Experience information" is information about jobs and projects that a candidate has been involved in in the past.

[0015] "Match level" is an evaluation of how well a candidate's characteristics and skills match the company's hiring requirements.

[0016] An "interview" is a procedure in which a company interacts directly with a candidate to assess their skills and attributes.

[0017] "Video" refers to visual data captured by a camera or other device.

[0018] "Audio" refers to auditory data recorded by a microphone or the like.

[0019] "Real time" is a temporal concept that refers to the immediate processing of current events.

[0020] "Analysis" is the process of breaking down data or information to make its contents easier to understand.

[0021] "Emotion analysis" is a technology that determines emotional states from video, audio, etc.

[0022] "Question Suggestions" are options that suggest appropriate questions to ask the candidate during the interview.

[0023] "Points of concern" are points that could become problems or matters that require attention.

[0024] "Scoring" is the process of scoring candidates based on evaluation criteria.

[0025] "Comparative evaluation" refers to the comparative analysis and evaluation of the scores and characteristics of multiple candidates.

[0026] A "report" is a document that summarizes the results of evaluation and analysis. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0028] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0029] First, the terms used in the following description will be explained.

[0030] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0031] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0032] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0033] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0034] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0035] [First embodiment]

[0036] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0037] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0038] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0039] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0040] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0041] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0042] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0043] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0044] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0045] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0046] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0047] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] System Overview

[0049] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatically creating job postings, analyzing candidates' characteristics and determining their match, analyzing real-time emotions during interviews, and scoring and comparative evaluation after interviews.

[0050] Program processing flow and specific examples

[0051] Automatic creation of job postings

[0052] 1. A user (company representative) logs in to a recruitment portal site and enters the recruitment requirements. For example, suppose the company is looking for a "software development engineer with at least three years of experience."

[0053] 2. The terminal sends the entered employment requirements to the server.

[0054] 3. The server inputs the received hiring requirements into the AI ​​model, which then generates the optimal job posting for the target and position based on past job postings and hiring history.

[0055] 4. The server sends the generated job posting to the terminal so that the user can check it. After checking, the user can make any necessary corrections and officially publish it.

[0056] Candidate characteristics analysis and match assessment

[0057] 1. The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[0058] 2. The terminal sends the uploaded document to the server.

[0059] 3. The server inputs the received document into a natural language processing (NLP) engine. The NLP engine analyzes the document and extracts skill and experience information. Examples of extracted skills include "financial industry" and "software development experience."

[0060] 4. The server uses the extracted information to evaluate the candidate's match using an AI model. The evaluation results are provided to the user (company representative), who determines whether the candidate has a high match.

[0061] Real-time analysis during the interview

[0062] 1. The user (interviewer) starts the interview and uses the video call system.

[0063] 2. The device transmits the interview video and audio data to the server in real time.

[0064] 3. The server uses a video analysis engine to analyze the candidate's emotions based on their facial expressions and tone of voice, for example, to determine their level of nervousness or confidence.

[0065] 4. The server displays the sentiment analysis results in real time on the interviewer's device and suggests suitable questions and areas of concern. For example, if the candidate is nervous, the server will prompt them to "Tell me more about your specific project experience."

[0066] Post-interview scoring and comparison

[0067] 1. After the interview is completed, the server integrates the collected video and audio data and the interviewer's evaluation.

[0068] 2. The server uses an AI model to score each candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[0069] 3. The server compares candidates based on the scoring results and determines the best candidate.

[0070] 4. The server generates a report containing the comparison results and sends it to the terminal. The user (company representative) makes the final hiring decision based on this report.

[0071] Technical effects

[0072] This system allows companies to streamline their recruitment processes and significantly reduce the burden on managers. In addition, AI-based data analysis enables objective and accurate candidate evaluation, enabling the rapid recruitment of suitable personnel. This is expected to improve a company's competitiveness and optimize the allocation of personnel.

[0073] The processing flow will be explained below.

[0074] Automatic creation of job postings

[0075] Step 1:

[0076] The user (company representative) logs in to the recruitment portal site and enters the recruitment requirements, for example, "Recruiting a software development engineer with at least three years of experience."

[0077] Step 2:

[0078] The terminal transmits the entered employment requirements to the server.

[0079] Step 3:

[0080] The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[0081] Step 4:

[0082] The AI ​​model generates optimal job descriptions, including requirements such as "Java or Python programming experience" and "team development experience."

[0083] Step 5:

[0084] The server sends the generated job posting to the terminal so that the user can check it.

[0085] Step 6:

[0086] After checking the job posting, the user makes any necessary corrections, makes a final check, and then publishes the job posting.

[0087] ---

[0088] Candidate characteristics analysis and match assessment

[0089] Step 1:

[0090] The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[0091] Step 2:

[0092] The terminal transmits the uploaded document to the server.

[0093] Step 3:

[0094] The server inputs the received document into a natural language processing (NLP) engine.

[0095] Step 4:

[0096] The NLP engine analyzes documents and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[0097] Step 5:

[0098] Based on the extracted information, the server uses an AI model to evaluate the candidate's match, for example, rating them as a "high match."

[0099] Step 6:

[0100] The server sends the evaluation results to the terminal so that the user (company representative) can check them.

[0101] ---

[0102] Real-time analysis during the interview

[0103] Step 1:

[0104] The user (interviewer) starts the interview and uses the video call system.

[0105] Step 2:

[0106] The terminal transmits the interview video and audio data to the server in real time.

[0107] Step 3:

[0108] The server uses a video analysis engine and an audio analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[0109] Step 4:

[0110] Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Please tell us more about your specific project experience."

[0111] Step 5:

[0112] The user (interviewer) conducts the interview based on the presented questions and asks additional questions if necessary.

[0113] ---

[0114] Post-interview scoring and comparison

[0115] Step 1:

[0116] The server integrates the collected video and audio data and the interviewer's evaluation after the interview is completed.

[0117] Step 2:

[0118] The server uses an AI model to score each candidate's match, for example, Candidate A might get 90 points, and Candidate B might get 85 points.

[0119] Step 3:

[0120] The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[0121] Step 4:

[0122] The server generates a report containing the comparison results and sends it to the terminal.

[0123] Step 5:

[0124] The user (company representative) makes the final hiring decision based on the received report.

[0125] ---

[0126] Through the above processing steps, the recruitment assistant AI system realizes an efficient and objective recruitment process, significantly reducing the burden on corporate managers.

[0127] Example 1

[0128] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0129] Traditional corporate recruitment processes require a wide range of manual tasks, such as creating job postings, evaluating candidates, conducting interviews, and scoring the results, which is time-consuming and labor-intensive. It is also difficult to analyze candidates' characteristics and emotions in real time, making objective evaluations difficult. Therefore, there is a need to reduce the burden on recruiters and achieve a more efficient and objective recruitment process.

[0130] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0131] In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents and extracting skills and experience information using a natural language processing engine, means for evaluating the candidate's match based on the skills and experience information, means for analyzing video and audio during the interview in real time and displaying the results of a sentiment analysis, means for presenting suitable question candidates and concerns during the interview, means for scoring the candidate's match after the interview, and means for comparatively evaluating multiple candidates based on the scoring results, thereby enabling efficiency improvement and objective evaluation of the entire hiring process.

[0132] "Employment requirements" refer to the specific conditions, such as skills, experience, educational background, and qualifications, that a company seeks in its employees.

[0133] A "job posting" is a document containing recruitment information published by a company, which lists the job types, conditions, and work content.

[0134] "Documents" refers to documents submitted by candidates, such as resumes and application forms.

[0135] A "natural language processing engine" is software or a system that analyzes the content of documents and extracts skill and experience information.

[0136] "Skills and experience information" is information that indicates the candidate's abilities and past work experience.

[0137] "Match" is an evaluation that indicates how closely a candidate's characteristics match the company's hiring requirements.

[0138] "Real-time analysis of video and audio" refers to a technology that analyzes video and audio collected during an interview on the spot.

[0139] "Emotion analysis results" are data showing the emotional state of a candidate obtained by analyzing their facial expressions and tone of voice.

[0140] "Suggested Questions and Concerns" present questions that interviewers should ask candidates and points to be aware of during the interview.

[0141] "Scoring" refers to the process of assigning a score to the interview results and the candidate's characteristics to evaluate them.

[0142] "Comparative evaluation" means comparing multiple candidates based on scoring results.

[0143] System Overview

[0144] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatically creating job postings, analyzing candidates' characteristics and determining their match, analyzing real-time emotions during interviews, and scoring and comparative evaluation after interviews.

[0145] Automatic creation of job postings

[0146] The user (company representative) accesses the login page using their company's account information and enters specific hiring requirements, such as "software development engineer with at least three years of experience." The device collects the information entered by the user in real time and sends it securely to the server using HTTPS. The server converts the received hiring requirements into an appropriate format and inputs it into an AI model using TensorFlow. The model references a past database and generates an optimal job posting. The server sends the generated job posting in JSON format to the device, which displays it on the user's screen. The user checks the content on the screen, modifies the job posting if necessary, and clicks the "Publish" button, which officially posts the job information on the portal site.

[0147] Examples:

[0148] If a user enters a job requirement for "an engineer with 5+ years of experience in front-end development," the server generates a corresponding job posting and provides it as "React.js developer with 5+ years of experience wanted." The user can then review the generated job posting, modify it, and publish it.

[0149] Example prompt:

[0150] "Please create a job posting for an engineer with at least 5 years of experience in front-end development."

[0151] Candidate characteristics analysis and match assessment

[0152] Users (candidates) upload their resumes and application forms to the system in an appropriate format (PDF or Word). The terminal scans the uploaded files and sends them to the server. The server launches a natural language processing (NLP) engine using PyTorch to analyze the text within the document. For example, it extracts keywords such as "financial industry" and "software development experience." The server inputs the extracted information into a pre-prepared evaluation algorithm and numerically evaluates the match between the candidate's skills and the company's requirements. The evaluation results are sent to the terminal, and the user can view a detailed evaluation of the candidate on the screen.

[0153] Examples:

[0154] If a candidate states that they have "software development experience in the financial industry" in the past, the server will extract "financial industry" and "software development experience" and compare them with the company's job requirements to evaluate the match as 90%.

[0155] Example prompt:

[0156] "Analyze the resumes of candidates with software development experience in the financial industry to determine the degree of match."

[0157] Real-time sentiment analysis during interviews

[0158] The user (interviewer) initiates a video call with a candidate via Zoom or Microsoft Teams. The device captures the video and audio data of the video call in real time and sends it to the server. The server runs a video analysis engine such as OpenCV to analyze the candidate's facial expressions and tone of voice. For example, it analyzes their level of nervousness and confidence. The server then creates candidate questions corresponding to the analysis results and displays them in real time on the interviewer's device. The interviewer can use this information to ask the candidate appropriate questions.

[0159] Examples:

[0160] If sentiment analysis reveals that a candidate is nervous during an interview, the server will display "High nervousness" and suggest questions to the interviewer, such as "Tell me more about your specific project experience."

[0161] Example prompt:

[0162] "Perform real-time sentiment analysis of candidate tension during interviews and suggest appropriate questions."

[0163] Post-interview scoring and comparison

[0164] The server combines the video and audio data collected during the interviews, as well as the interviewers' manual evaluations, and stores them as a single dataset. The server inputs the combined dataset into an AI model, which evaluates each candidate's skills, aptitude, and expressions during the interview to create a score. For example, candidate A may receive a score of 90, while candidate B receives a score of 85. The server compares the scoring results and determines the most suitable candidate, taking into account algorithms and past hiring performance data. The server generates a detailed report including the final evaluation and comparison results and sends it to the device. The user makes the final hiring decision based on this report. The user reviews the report displayed on the device and selects the most suitable candidate. Once the final decision is made, the server sends an offer of employment.

[0165] Technical effects

[0166] This system allows companies to streamline their recruitment processes and significantly reduce the burden on managers. In addition, AI-based data analysis enables objective and accurate candidate evaluation, enabling the rapid recruitment of suitable personnel. This is expected to improve a company's competitiveness and optimize the allocation of personnel.

[0167] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0168] Step 1:

[0169] A user logs in to a recruitment portal site and enters their recruitment requirements.

[0170] Input: User inputs recruitment requirements (e.g., "Software development engineer with at least 3 years of experience")

[0171] What happens: The user logs in using their company's account information and enters their job requirements in the text fields.

[0172] Step 2:

[0173] The terminal transmits the entered employment requirements to the server.

[0174] Input: Recruitment requirements data

[0175] Data processing / calculation: The device collects input information in real time and sends it securely to the server using HTTPS

[0176] Output: Recruitment requirements data sent to the server

[0177] Specific operation: The terminal registers data in the transmission queue and sends it to the server via HTTPS communication.

[0178] Step 3:

[0179] The server inputs the recruitment requirements received into the AI ​​model and generates a job posting.

[0180] Input: Recruitment requirements data received by the server

[0181] Data processing / calculation: The server converts the data into the appropriate format, inputs it into an AI model using TensorFlow, and generates a job posting.

[0182] Output: Generated job posting data

[0183] Specific operation: The server retrieves relevant information from the database and inputs it into the AI ​​model to generate a job posting.

[0184] Step 4:

[0185] The server sends the generated job posting to the terminal and the user confirms it.

[0186] Input: Generated job posting data

[0187] Data processing / calculation: The server converts the generated job posting into JSON format and sends it to the terminal.

[0188] Output: The job posting displayed to the user

[0189] Specific operation: The server sends JSON format data to the terminal, which parses it and displays it on the user's screen.

[0190] Step 5:

[0191] The user modifies and publishes the job posting.

[0192] Input: User-modified job posting data

[0193] Data processing / calculation: The user edits the job posting details on the screen and clicks the "Publish" button.

[0194] Output: Published job postings

[0195] Specific operation: The user checks the content, makes corrections, and then presses the publish button, and the job information is posted on the portal site.

[0196] Step 6:

[0197] The user (candidate) uploads their resume and application form.

[0198] Input: Candidates upload their resumes and application forms

[0199] Data processing / calculation: Upload document data to the system

[0200] Output: Uploaded document data

[0201] What happens: Candidates follow the system instructions and upload documents using the file selection button.

[0202] Step 7:

[0203] The terminal sends the uploaded document to the server.

[0204] Input: Uploaded document data

[0205] Data processing / calculation: The device scans the file and sends it to the server

[0206] Output: Document data sent to the server

[0207] Specific operation: The device temporarily stores the uploaded file and securely sends it to the server.

[0208] Step 8:

[0209] The server inputs the document into an NLP engine for analysis.

[0210] Input: Document data sent to the server

[0211] Data processing / calculation: The server runs a natural language processing (NLP) engine using PyTorch to perform analysis. Specifically, it extracts keywords such as "financial industry" and "software development experience."

[0212] Output: Parsed skills and experience information

[0213] How it works: The server inputs documents into the NLP engine and extracts important skills and experience information.

[0214] Step 9:

[0215] The server uses the analysis results to evaluate the degree of match using an AI model.

[0216] Input: Parsed skills and experience information

[0217] Data processing / calculation: The server inputs the extracted information into an evaluation algorithm and evaluates the candidate's match numerically.

[0218] Output: Evaluation results

[0219] Specific operation: The server scores the extracted information based on an evaluation algorithm.

[0220] Step 10:

[0221] The server provides the evaluation results to the user.

[0222] Input: Evaluation result data

[0223] Data processing / calculation: Evaluation results sent to the terminal

[0224] Output: Evaluation results displayed on the user's screen

[0225] Specific operation: The server sends the evaluation results in JSON format to the terminal, and the user confirms them.

[0226] Step 11:

[0227] The user (interviewer) starts the interview using the video call system.

[0228] Input: Start the video call system

[0229] Data processing / calculation: Capture of interview video and audio data

[0230] Output: Captured video and audio data

[0231] What happens: The interviewer will start a video call using Zoom or Microsoft Teams.

[0232] Step 12:

[0233] The terminal transmits the interview video and audio data to the server.

[0234] Input: Captured video and audio data

[0235] Data processing / calculation: Send to server in real time

[0236] Output: Video and audio data sent to the server

[0237] Specific operation: The device captures video and audio data in real time and sends it to the server.

[0238] Step 13:

[0239] The server performs emotion analysis using a video analysis engine.

[0240] Input: Video and audio data sent to the server

[0241] Data processing / calculation: Emotion analysis is performed using OpenCV etc. to analyze tension and confidence.

[0242] Output: Sentiment analysis results

[0243] Specific operation: The server inputs video and audio data into an analysis engine to evaluate the emotional state.

[0244] Step 14:

[0245] The server displays the analysis results and suitable question candidates on the device.

[0246] Input: Sentiment analysis results

[0247] Data processing / calculation: Generate suitable question candidates and send them to the device

[0248] Output: Question candidates and sentiment analysis results displayed on the user's screen

[0249] Specific operation: The server generates candidate questions based on the analysis results and sends them to the device, which then displays them to the user.

[0250] Step 15:

[0251] The server consolidates the collected data.

[0252] Input: Video and audio data collected during the interview, and manual assessment by the interviewer

[0253] Data processing / calculation: Data integration and storage

[0254] Output: A consolidated dataset

[0255] Specific operation: The server consolidates and centrally manages all data.

[0256] Step 16:

[0257] The server scores each candidate using an AI model.

[0258] Input: Integrated dataset

[0259] Data processing / calculation: Input data into the AI ​​model to generate a score for each candidate (e.g., candidate A scores 90, candidate B scores 85)

[0260] Output: Scoring results

[0261] How it works: The server inputs data into the AI ​​model and generates a score.

[0262] Step 17:

[0263] The server compares the candidates to determine the best candidate.

[0264] Input: Scoring results

[0265] Data processing / calculation: Use comparison algorithms to determine the best candidates

[0266] Output: The best candidate is identified

[0267] Specific operation: The server compares the scoring results and selects the most suitable candidate.

[0268] Step 18:

[0269] The server sends the comparison results as a report to the terminal.

[0270] Input: The best candidate's results

[0271] Data processing / calculation: Generate detailed reports and send them to your device

[0272] Output: The report that is displayed on the user's screen

[0273] Specific operation: The server generates a report based on the evaluation results and sends it to the terminal, which then displays it to the user.

[0274] Step 19:

[0275] The user makes the final hiring decision based on the report.

[0276] Input: Best candidate report

[0277] Data processing / calculation: Making decisions based on reports

[0278] Output: Final hiring decision

[0279] Specific actions: The user reviews the report, selects the best candidates, and sends out job offers.

[0280] (Application example 1)

[0281] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0282] In the recruitment process, analyzing candidate characteristics and emotions during interviews is important, but there is no system that utilizes this data in real time to evaluate risk factors. As a result, companies are forced to rely on subjective judgment when selecting appropriate candidates, making efficient recruitment difficult. In addition, it is difficult to detect security risks early in post-recruitment activities.

[0283] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0284] In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and displaying the emotion analysis results, means for analyzing the candidate's behavior in real time during the interview and performing a risk assessment, means for scoring the candidate's match after the interview, and means for comparatively assessing multiple candidates based on the scoring results. This enables a more objective and efficient hiring process based on real-time emotion and risk analysis.

[0285] "Means for inputting a company's recruitment requirements" refers to an interface that allows a company to input the conditions, qualifications, skills, etc. of the personnel it wishes to hire into the system.

[0286] The "means for automatically generating a job posting based on the hiring requirements" refers to an algorithm and system that creates optimal job advertisements and job postings based on the hiring requirements entered.

[0287] "Means for receiving documents submitted by candidates" refers to a function for receiving resumes and application forms uploaded by candidates to the system.

[0288] The "means for analyzing the document and extracting skill and experience information" is a function that uses natural language processing technology to extract information such as the candidate's skills and work history from the received document.

[0289] The "means for evaluating the degree of match of a candidate based on the skills and experience information" refers to an algorithm and system that compares the extracted skills and experience information and evaluates the degree of suitability of a candidate for the recruitment requirements.

[0290] "A means of analyzing video and audio during interviews in real time and displaying the results of emotional analysis" is a function that instantly processes video and audio collected during interviews, analyzes the candidate's facial expressions and tone of voice, and displays their emotional state.

[0291] "Means for analyzing candidate behavior in real time during interviews and conducting risk assessments" refers to a function that observes candidate actions and behaviors occurring during interviews in real time and assesses risk based on that information.

[0292] "Means for scoring candidate match after interview" refers to algorithms and systems that quantify and evaluate a candidate's suitability based on data collected after the interview.

[0293] The "means for comparatively evaluating a plurality of candidates based on the scoring results" is a function for comparing the evaluation results of scored candidates and selecting the most suitable candidate.

[0294] This invention provides a recruitment assistant AI system that integrates multiple functions to streamline a company's recruitment process and reduce the burden on managers. The configuration and operation of this system are described below.

[0295] Automatic creation of job postings

[0296] The server automatically generates job postings based on information received through the company's recruitment requirements input method. At this time, the server uses a generative AI model based on past recruitment data and recruitment performance to create the optimal job posting. The user can check this job posting via their device, make any necessary corrections, and publish the job information.

[0297] Candidate characteristics analysis and match assessment

[0298] The server receives the resumes and application forms submitted by candidates. Based on this, it analyzes the documents using natural language processing (NLP) technology to extract skill and experience information. The server then inputs the extracted information into a generative AI model to evaluate the candidate's match.

[0299] Real-time sentiment analysis and risk assessment during interviews

[0300] The user (interviewer) uses smart glasses during the interview to collect video and audio data. The server analyzes this data in real time, analyzing the candidate's facial expressions and tone of voice to understand their emotional state. It also analyzes the candidate's behavior in real time during the interview and performs a risk assessment. This data is displayed on the interviewer's device and serves as a reference for how to respond.

[0301] Post-interview scoring and comparative evaluation

[0302] After the interviews are completed, the server integrates the collected data and scores the candidates' match. It uses a generative AI model to evaluate each candidate's suitability and compare multiple candidates. It then generates a report of the best candidates and provides it to the recruiter.

[0303] Hardware and software used

[0304] This system mainly uses the following hardware and software:

[0305] Hardware: Smart glasses, devices (PC, tablet, etc.)

[0306] Software: Python, OpenCV, Keras, NLP engine, generative AI model

[0307] Examples of specific examples and prompts

[0308] For example, if a company is looking for a "software development engineer with at least three years of experience," the server will generate the most suitable job posting based on this information and analyze the candidate's submitted documents. During the interview, the smart glasses will analyze the candidate's behavior in real time and perform a risk assessment.

[0309] Example prompt sentence:

[0310] "Develop an application that uses camera footage to analyze candidates' facial expressions in real time, determine their emotions, and conduct a risk assessment. Display the results in real time."

[0311] In this way, the present invention assists in the entire recruitment process, allowing companies to efficiently recruit the right talent.

[0312] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0313] Step 1:

[0314] The user (company representative) enters the recruitment requirements.

[0315] Input: Job requirements (e.g., "Software development engineer with at least 3 years of experience")

[0316] Output: The entered recruitment requirements are sent to the server.

[0317] Specific operations: The company representative logs in to the recruitment portal site and enters the necessary recruitment requirements in detail into the form.

[0318] Step 2:

[0319] The server automatically generates a job posting based on the received hiring requirements.

[0320] Input: Recruitment requirements

[0321] Output: Generated job posting

[0322] Specific operation: The server uses the recruitment requirement data input into the generative AI model, refers to past cases, and creates the most appropriate job posting.

[0323] Step 3:

[0324] The server sends the generated job posting to the user's (company representative's) terminal.

[0325] Input: Generated job posting

[0326] Output: Job posting confirmation screen

[0327] Specific operation: The server sends the created job posting to the company's employee's terminal and displays a screen where the employee can check and edit the job posting.

[0328] Step 4:

[0329] The user uploads the candidate's documents (resume and application form).

[0330] Input: resume and application form

[0331] Output: Uploaded documents are sent to the server.

[0332] What happens: Candidates use a web portal to upload their resume and application form.

[0333] Step 5:

[0334] The server analyzes the uploaded documents using a natural language processing (NLP) engine.

[0335] Input: resume and application form

[0336] Output: Extraction results of skills and experience information

[0337] Specific operation: The server uses an NLP engine to analyze the contents of the document and extract skill and experience information such as "software development experience in the financial industry."

[0338] Step 6:

[0339] The server evaluates the match of the candidate based on the extracted information.

[0340] Input: Extracted skills and experience information

[0341] Output: Candidate match rating

[0342] How it works: The server uses a generative AI model to match the extracted information with the job requirements and score the match.

[0343] Step 7:

[0344] The user (interviewer) uses smart glasses during the interview to collect video and audio of the candidate in real time.

[0345] Input: Video and audio data from the interview

[0346] Output: Real-time data sent to the server

[0347] How it works: The interviewer puts on the smart glasses and begins the interview. The glasses' camera and microphone collect video and audio of the candidate in real time.

[0348] Step 8:

[0349] The server analyzes the data received in real time and displays an assessment of emotional state and behavioral risk.

[0350] Input: Video and audio data from the interview

[0351] Output: Real-time display of emotional state and risk assessment

[0352] Specific operation: The server inputs the received data into an analysis engine, analyzes the candidate's emotional state (e.g., level of tension, confidence), and displays the evaluation results on the interviewer's terminal.

[0353] Step 9:

[0354] The server consolidates the data collected after the interview and scores the candidate's match.

[0355] Input: Video and audio data collected during the interview, and interviewer evaluation

[0356] Output: Match score

[0357] How it works: The server consolidates the collected data and uses a generative AI model to score candidates for match.

[0358] Step 10:

[0359] The server performs comparative evaluation based on the scoring results of multiple candidates and generates a report.

[0360] Input: Match score for each candidate

[0361] Output: Comparative evaluation report

[0362] Specific operation: The server compares the scoring results of each candidate, generates a report to select the most suitable candidate, and provides it to the recruiter.

[0363] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0364] System Overview

[0365] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. In particular, by incorporating an emotion engine, it enables more precise analysis that takes into account the candidate's emotional state during and after the interview. This system supports each stage of the recruitment process, including automatic creation of job postings, analysis of candidate characteristics and match determination, real-time emotion analysis using the emotion engine, and post-interview scoring and comparative evaluation.

[0366] Program processing flow and specific examples

[0367] Automatic creation of job postings

[0368] 1. A user (company representative) logs in to a recruitment portal site and enters the recruitment requirements. For example, they might enter "Recruiting a software development engineer with at least three years of experience."

[0369] 2. The terminal sends the entered employment requirements to the server.

[0370] 3. The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[0371] 4. The AI ​​model generates the most suitable job description, including requirements such as "Java and Python programming experience" and "team development experience."

[0372] 5. The server sends the generated job posting to the terminal so that the user can view it.

[0373] 6. After checking the job posting, the user will make any necessary corrections, make a final confirmation, and then publish it.

[0374] Candidate characteristics analysis and match assessment

[0375] 1. The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience developing software in the financial industry."

[0376] 2. The terminal sends the uploaded document to the server.

[0377] 3. The server inputs the received document into a natural language processing (NLP) engine.

[0378] 4. The NLP engine analyzes the document and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[0379] 5. Based on the extracted information, the server uses an AI model to evaluate the candidate's match, e.g., a "high match."

[0380] 6. The server sends the evaluation results to the terminal so that the user (company representative) can check them.

[0381] Real-time analysis during the interview

[0382] 1. The user (interviewer) starts the interview and uses the video call system.

[0383] 2. The device transmits the interview video and audio data to the server in real time.

[0384] 3. The server uses a video analysis engine and a voice analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[0385] 4. The server uses an emotion engine to perform a more detailed analysis of the candidate's emotional state, assessing their level of nervousness, confidence, etc.

[0386] 5. Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Tell me more about your specific project experience."

[0387] 6. The user (interviewer) conducts the interview based on the questions presented and asks additional questions if necessary.

[0388] Post-interview scoring and comparison

[0389] 1. The server integrates the video and audio data collected after the interview and the interviewer's evaluation.

[0390] 2. The server uses an AI model to score each candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[0391] 3. The server adjusts the scoring results based on the emotion data obtained by the emotion engine, and calculates the final score taking into account the level of tension and confidence from the emotion data.

[0392] 4. The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[0393] 5. The server generates a report containing the comparison results and sends it to the terminal.

[0394] 6. The user (company representative) makes the final hiring decision based on the received report.

[0395] Technical effects

[0396] This system enables companies to streamline their recruitment processes and significantly reduce the burden on managers. In particular, by analyzing emotional data using an emotion engine, it is possible to grasp the true state of candidates during interviews, enabling more accurate evaluations. This will enable companies to quickly hire the right talent, which is expected to improve a company's competitiveness and optimize personnel allocation.

[0397] The processing flow will be explained below.

[0398] Automatic creation of job postings

[0399] Step 1:

[0400] The user (company representative) logs in to the recruitment portal site and enters the recruitment requirements, for example, "Recruiting a software development engineer with at least three years of experience."

[0401] Step 2:

[0402] The terminal transmits the entered employment requirements to the server.

[0403] Step 3:

[0404] The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[0405] Step 4:

[0406] The AI ​​model generates optimal job postings, such as those that include requirements such as "Java or Python programming experience" and "team development experience."

[0407] Step 5:

[0408] The server sends the generated job posting to the terminal, where it is displayed so that the user can check it.

[0409] Step 6:

[0410] The user checks the generated job posting, modifies it as necessary, and then publishes it.

[0411] ---

[0412] Candidate characteristics analysis and match assessment

[0413] Step 1:

[0414] Users (candidates) upload their resumes and application forms to a recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[0415] Step 2:

[0416] The terminal transmits the uploaded document to the server.

[0417] Step 3:

[0418] The server inputs the received document into a natural language processing (NLP) engine.

[0419] Step 4:

[0420] The NLP engine analyzes documents and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[0421] Step 5:

[0422] Based on the extracted information, the server uses an AI model to evaluate the candidate's match, for example, rating them as a "high match."

[0423] Step 6:

[0424] The server sends the evaluation results to the terminal and displays them so that the user (company representative) can check them.

[0425] ---

[0426] Real-time analysis during the interview

[0427] Step 1:

[0428] The user (interviewer) starts the interview and uses the video call system.

[0429] Step 2:

[0430] The terminal transmits the interview video and audio data to the server in real time.

[0431] Step 3:

[0432] The server uses a video analysis engine and an audio analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[0433] Step 4:

[0434] The server uses an emotion engine to further analyze the candidate's emotional state, assessing, for example, their level of nervousness and confidence.

[0435] Step 5:

[0436] Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Please tell us more about your specific project experience."

[0437] Step 6:

[0438] The user (interviewer) conducts the interview based on the presented questions and asks additional questions as necessary.

[0439] ---

[0440] Post-interview scoring and comparison

[0441] Step 1:

[0442] The server integrates the collected video and audio data and the interviewer's evaluation after the interview is completed.

[0443] Step 2:

[0444] The server uses an AI model to score each candidate's match, for example, Candidate A might get 90 points, and Candidate B might get 85 points.

[0445] Step 3:

[0446] The server adjusts the scoring results based on the emotion data obtained by the emotion engine, and calculates a final score that takes into account the level of tension and confidence from the emotion data.

[0447] Step 4:

[0448] The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[0449] Step 5:

[0450] The server generates a report containing the comparison results and sends it to the terminal.

[0451] Step 6:

[0452] The user (company representative) makes the final hiring decision based on the received report.

[0453] ---

[0454] Through the above processing steps, this system streamlines the entire recruitment process, reducing the burden on managers, and improves the accuracy of candidate aptitude assessment through detailed emotion analysis using an emotion engine.

[0455] Example 2

[0456] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0457] Traditional recruitment processes are time-consuming and labor-intensive, and rely on human evaluation, making them difficult to operate efficiently, especially when there are a large number of applicants. It is also difficult to determine a candidate's actual emotional state during the interview, making evaluations subjective. This increases the risk of delays in hiring the right talent and increases the likelihood of mismatches. Therefore, there is a need for a way to streamline the entire recruitment process and improve accuracy.

[0458] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0459] In this invention, the server includes means for inputting hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and providing emotion analysis results, means for scoring the candidate's match after the interview, means for comparatively evaluating multiple candidates based on the scoring results, means for adjusting the scoring results using emotion data, and means for selecting the most suitable candidate based on the adjusted scoring results. This makes the entire hiring process more efficient and enables more accurate and objective evaluations.

[0460] The "means for inputting recruitment requirements" is a method for providing an interface for users to input the conditions and needs of a company regarding recruitment.

[0461] "Means for automatically generating a job posting based on the hiring requirements" refers to a method for automatically creating an appropriate job posting using an AI model based on the hiring requirements entered.

[0462] The "means for receiving documents submitted by candidates" refers to a method for receiving documents such as resumes and application forms uploaded by candidates onto a server.

[0463] The "means for analyzing the document and extracting skill and experience information" is a method for analyzing the received document using natural language processing technology and extracting information such as the candidate's skills and work experience.

[0464] "Means for evaluating the degree of match of a candidate based on the skills and experience information" refers to a method for evaluating the degree to which a candidate matches the company's recruitment requirements based on the extracted skills and experience information.

[0465] "Means for analyzing video and audio data during an interview in real time and providing emotional analysis results" refers to a method for analyzing video and audio data acquired during an interview in real time, determining the emotional state of the candidate, and providing the results to the interviewer.

[0466] "Method of scoring candidate match after interview" is a method of scoring each candidate's suitability based on interview data.

[0467] The "means for comparatively evaluating a plurality of candidates based on the scoring results" refers to a method for comparing a plurality of candidates based on the scoring results and selecting the candidate with the highest degree of suitability.

[0468] "Means for adjusting scoring results using emotional data" refers to a method for correcting scoring results based on emotional data obtained during the interview and evaluating the final degree of fit.

[0469] The "means for selecting the most suitable candidate based on the adjusted scoring results" is a method for selecting the most suitable candidate based on the adjusted scoring results.

[0470] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatic creation of job postings, candidate characteristic analysis and match assessment, real-time emotion analysis using an emotion engine, and post-interview scoring and comparative evaluation. In particular, the incorporation of an emotion engine enables more precise analysis that takes into account the candidate's emotional state during and after the interview.

[0471] Hardware and software used

[0472] The server will be a high-performance cloud server. Specifically, cloud platforms such as Amazon Web Services (AWS) and Microsoft Azure are considered. Standard PCs, tablets, and smartphones can be used as terminals. The following software will be used:

[0473] Natural language processing model (NLP engine): SpaCy or Stanford NLP

[0474] Generative AI models: GPT-3 and BERT

[0475] Video analysis engine: OpenCV

[0476] Voice analysis engine: Librosa

[0477] Sentiment analysis engine: AI model equipped with FaceAPI and Sentiment Analysis

[0478] Video calling system: Zoom or Microsoft Teams

[0479] Program processing

[0480] Automatic creation of job postings

[0481] The user (company representative) logs in to the recruitment portal site and enters their hiring requirements. For example, they may enter requirements such as "We are looking for a software development engineer with at least three years of experience." The device then sends the entered data to the server. The server then inputs the received data into a generative AI model, compares it with past job posting data, and generates the most suitable job posting. The generated job posting is then sent to the device, where it can be viewed by the user. Any necessary corrections are made, and the job posting is published after a final confirmation.

[0482] Candidate characteristics analysis and match assessment

[0483] The user (candidate) uploads their resume and application form to a recruitment portal site. For example, they may upload a document stating that they have previous experience developing software in the financial industry. The device then sends the uploaded document to the server. The server inputs the received document into an NLP engine, analyzes the document, and extracts skill and experience information. Based on the extracted information, an AI model is used to evaluate the candidate's match. The evaluation results are sent to the device and can be viewed by the user.

[0484] Real-time analysis during the interview

[0485] The user (interviewer) begins the interview using a video call system. The device sends the interview video and audio data to the server in real time. The server uses a video analysis engine and an audio analysis engine to analyze the video and audio data in real time. The emotion analysis engine is used to perform a detailed analysis of the candidate's emotional state and evaluate their level of nervousness and confidence. The analysis results are displayed on the interviewer's device as suitable question suggestions and areas of concern. For example, if the candidate is nervous, a question such as "Tell me more about your specific project experience" will be presented.

[0486] Post-interview scoring and comparative evaluation

[0487] Once the interview is over, the server combines the collected video and audio data and the interviewer's evaluation. It then uses an AI model to score each candidate's match. For example, candidate A might be rated 90 points, and candidate B 85 points. The scoring results are then adjusted based on emotional data to calculate the final score. Multiple candidates are compared and evaluated to determine the most suitable candidate. A report containing the final comparison results is generated and sent to the device. The user (company representative) makes the final hiring decision based on this report.

[0488] Examples of concrete examples and prompts

[0489] For example, use the following prompt:

[0490] 1. "We are looking for a software development engineer with at least 3 years of experience. Experience with Java or Python programming and team development is essential."

[0491] 2. "I have previous software development experience in the financial industry."

[0492] 3. "Tell me more about your specific project experience."

[0493] This will enable companies to streamline the recruitment process and significantly reduce the burden on managers. In particular, analyzing emotional data using an emotion engine will enable candidates to understand their true state during interviews, enabling more accurate evaluations.

[0494] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0495] Step 1:

[0496] A user logs in to a recruiting portal site.

[0497] Specific operation: The user accesses the login screen of the portal site using their company account and enters their authentication information. The server verifies the entered authentication information and authenticates the user. If authentication is successful, the user can access the main screen of the recruitment portal site.

[0498] Step 2:

[0499] The user enters the hiring requirements.

[0500] Input: Requirements such as job type, required skills, years of experience, etc.

[0501] Specific operation: The user enters specific employment requirements into a form on the site, such as "Seeking a software development engineer with at least three years of experience. Experience in Java or Python programming and team development is essential." The terminal then displays the entered information and allows the user to confirm it.

[0502] Step 3:

[0503] The terminal transmits the input data to the server.

[0504] Input: User-entered recruitment requirements

[0505] Output: Requirements data sent to the server

[0506] Specific operation: The terminal sends the recruitment requirement data to the server using an HTTP POST request. The server parses the received data and stores it in an appropriate format.

[0507] Step 4:

[0508] The server inputs the data into a generative AI model for analysis.

[0509] Input: Recruitment requirements data

[0510] Output: Generated job posting

[0511] How it works: The server inputs the received recruitment requirements data into a natural language processing model such as GPT-3 or BERT. The AI ​​model then references past recruitment data and generates a job posting that best fits the input requirements.

[0512] Step 5:

[0513] The server sends the generated job posting to the terminal.

[0514] Input: Generated job posting

[0515] Output: Job posting displayed on terminal

[0516] Specific operation: The server sends the generated job posting data to the terminal so that the user can check it. The terminal displays the received data and provides an interface that allows the user to check and modify it.

[0517] Step 6:

[0518] The user reviews and publishes the job posting.

[0519] Specific operation: The user checks the job posting displayed on the device and makes any necessary corrections. After completing the final check, the user presses the "Publish" button to publish the job posting. The device then sends the published information to the server and publishes the job posting.

[0520] Step 7:

[0521] The user uploads a resume or application form.

[0522] Input: resume and application form

[0523] Specific operation: The user (candidate) uses the upload screen on the recruitment portal site to select and upload the necessary documents such as resumes and application forms. The terminal then sends the selected documents to the server.

[0524] Step 8:

[0525] The terminal sends the document to the server.

[0526] Input: Uploaded document

[0527] Output: Document data stored on the server

[0528] Specific operation: The terminal sends the document data uploaded by the user to the server, and the server stores the received data in an appropriate format.

[0529] Step 9:

[0530] The server inputs the document into the NLP engine.

[0531] Input: Uploaded document data

[0532] Output: Skills and experience information

[0533] How it works: The server inputs the received document data into a natural language processing engine such as SpaCy or Stanford NLP for analysis. The NLP engine extracts keywords related to skills and work experience from the document.

[0534] Step 10:

[0535] The server evaluates the match of the candidate based on the extracted information.

[0536] Input: Skills and experience information

[0537] Output: Candidate match evaluation results

[0538] Specific operation: The server inputs the skills and experience information extracted by the NLP engine into the generative AI model to evaluate the candidate's match with the recruitment requirements. The evaluation result (e.g., "high match") is scored based on the company's recruitment criteria.

[0539] Step 11:

[0540] The server transmits the evaluation results to the terminal.

[0541] Input: Match evaluation result

[0542] Output: Evaluation results displayed on the terminal

[0543] Specific operation: The server sends the evaluation results to the terminal so that the user can check them. The terminal displays the received evaluation results and notifies the user.

[0544] Step 12:

[0545] The user starts the interview.

[0546] Specific operation: The user (interviewer) starts an interview with a candidate using a video call system such as Zoom or Microsoft Teams. The user accesses the video call system from their device and invites the candidate to participate.

[0547] Step 13:

[0548] The device transmits the interview video and audio data to the server in real time.

[0549] Input: Video and audio data from the interview

[0550] Output: Real-time data sent to the server

[0551] Specific operation: The device streams video and audio data captured during the interview to the server in real time, and the server prepares the infrastructure to process the received data appropriately.

[0552] Step 14:

[0553] The server analyzes the data and uses an emotion engine to analyze the emotional state.

[0554] Input: Real-time video and audio data

[0555] Output: Analysis of the candidate's emotional state

[0556] Specific operation: The server analyzes video and audio data using a video analysis engine (OpenCV) and an audio analysis engine (Librosa). The server inputs the analysis results into an emotion engine, which analyzes the candidate's facial expressions and tone of voice in real time.

[0557] Step 15:

[0558] The server sends potential questions and concerns to the device.

[0559] Input: Sentiment analysis results

[0560] Output: Possible questions and concerns displayed on the device

[0561] Specific operation: The server generates suitable candidate questions and concerns based on the results of sentiment analysis. The server then sends the generated information to the terminal in real time and presents it to the user (interviewer).

[0562] Step 16:

[0563] The user conducts the interview based on the questions presented.

[0564] Specific operation: The user conducts the interview while referring to the candidate questions and concerns displayed on the device. If necessary, the user asks additional questions and collects the candidate's answers.

[0565] Step 17:

[0566] The server consolidates the data after the interview is completed.

[0567] Input: Interview video, audio data, interviewer notes

[0568] Output: Consolidated interview data

[0569] Specific operation: After the interview is completed, the server integrates the collected video and audio data, as well as the interviewer's evaluation notes. The server then integrates and centrally manages this data.

[0570] Step 18:

[0571] The server uses an AI model to score each candidate's match.

[0572] Input: Consolidated interview data

[0573] Output: Match score for each candidate

[0574] Specific operation: The server inputs the integrated interview data into the generative AI model and scores each candidate's match. For example, candidate A might receive a score of 90, and candidate B might receive a score of 85.

[0575] Step 19:

[0576] The server adjusts the scoring results.

[0577] Input: Emotion data, scoring results

[0578] Output: Adjusted score

[0579] How it works: The server modifies the scoring results based on emotional data obtained during the interview, for example, adjusting the final score to take into account the level of nervousness or confidence.

[0580] Step 20:

[0581] The server compares and evaluates multiple candidates.

[0582] Input: Adjusted score

[0583] Output: Evaluation results for the best candidates

[0584] Specific operation: The server compares and evaluates multiple candidates based on the adjusted scores. The server selects the candidate with the highest fit.

[0585] Step 21:

[0586] The server generates a report and sends it to the device.

[0587] Input: Evaluation results of the best candidate

[0588] Output: Generated report

[0589] Specific operation: The server automatically generates a report summarizing the evaluation results in detail. The generated report is sent from the server to the terminal so that the user can check it.

[0590] Step 22:

[0591] The user makes the final hiring decision.

[0592] Specific operation: The user checks the report displayed on the terminal and makes a final hiring decision. If necessary, a final interview or additional checks are conducted, and then a formal hiring decision is issued.

[0593] This will streamline the entire hiring process and allow for more accurate and objective evaluations.

[0594] (Application example 2)

[0595] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0596] The recruitment process in a company is extremely important, especially in the security department, and requires accurate assessment of candidates' reliability and suitability. However, this process places a heavy burden on recruiters, making it difficult to secure the right talent. It is also difficult to grasp a candidate's emotional state in real time and make an accurate assessment. To solve these issues, a system is needed to streamline the entire recruitment process and assess candidates' reliability and nervousness.

[0597] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and displaying the emotion analysis results, means for scoring the candidate's match after the interview, means for comparatively evaluating multiple candidates based on the scoring results, means for efficiently securing personnel within the security department, and an emotion engine for evaluating the candidate's reliability and level of tension. This improves the efficiency of the hiring process in the security department and enables the prompt securing of suitable personnel.

[0598] An "enterprise" is a legal entity or organization established to carry out economic activities as an organization and provide goods and services.

[0599] "Recruitment requirements" refer to the standards, such as skills, experience, and qualifications, that a company requires of new employees.

[0600] A "job posting" is a document that provides information such as the type of job and conditions that a company is recruiting for to job seekers.

[0601] A "candidate" is a job seeker who applies for a job at a company and seeks employment.

[0602] "Document" refers to information in written form, such as a resume or curriculum vitae submitted by a candidate.

[0603] "Skills" refer to the knowledge and abilities required to efficiently carry out specific tasks or operations.

[0604] "Experience information" refers to specific information about the job or project a candidate has previously undertaken.

[0605] "Match rate" is an indicator that shows how closely a candidate's skills and experience match the company's hiring requirements.

[0606] "Footage" refers to video data that captures the candidate's appearance and behavior during an interview.

[0607] "Audio" refers to audio data recorded from the candidate's voice during the interview.

[0608] "Emotion analysis results" refer to the evaluation results of the candidate's emotional state obtained by analyzing video and audio data.

[0609] "Scoring" is the process of quantifying and evaluating a candidate's match.

[0610] "Comparative evaluation" is the process of evaluating multiple candidates based on their scoring results and selecting the most suitable candidate.

[0611] "Human resource acquisition" refers to a series of activities that a company undertakes to properly recruit and secure the human resources it needs.

[0612] "Credibility" is an indicator of whether a candidate's words, actions, and behavior are trustworthy.

[0613] "Tension level" is an indicator of the level of stress and anxiety a candidate feels during an interview.

[0614] An "emotion engine" is software or algorithms that analyze video and audio data to assess a candidate's emotional state.

[0615] This invention provides a "security interview assistant" system that enables companies to efficiently and accurately select the personnel they are looking for. The system for realizing this application example will be described in detail below.

[0616] First, the user (company recruiter) logs in to the recruitment portal site and enters the company's recruitment requirements. The entered recruitment requirements are then sent to the server via the terminal. The generative AI model on the server automatically generates a job posting based on these recruitment requirements.

[0617] Candidates upload documents such as their resumes and work histories to a recruitment portal site. These documents are then sent to a server via their device, where the server's natural language processing engine analyzes the documents to extract information about skills and experience. Based on this information, the candidate's match is evaluated.

[0618] When conducting a real-time interview, the user (interviewer) talks with the candidate via a video chat system. The device sends video and audio data to the server in real time. The server's video and audio analysis engines analyze the candidate's facial expressions and tone of voice, and use an emotion engine to assess their level of nervousness and confidence. This allows the interviewer to be presented with suitable question candidates and areas of concern.

[0619] Once the interview is over, the server combines the collected video and audio data, as well as the interviewer's evaluation, and uses an AI model to score each candidate's match. The scoring results are then adjusted, taking into account emotional data, to calculate a final score. Based on the scoring results, the server compares and evaluates multiple candidates, generates a report to select the most suitable candidate, and sends it to the device.

[0620] Specific applications include:

[0621] 1. Automatic job posting generation:

[0622] The server generates the most suitable job posting based on the security-related recruitment requirements entered by users who log in to the recruitment portal site. Example: Automatically generate a job posting for "candidates with specialized knowledge and work experience in network security with at least three years."

[0623] 2. Candidate characteristics analysis and match assessment:

[0624] The server's natural language processing engine analyzes the resumes and CVs uploaded by candidates, extracts security-related skills and experience, and determines the degree of match. For example, keywords such as "CISSP certification" and "security audit experience" are extracted.

[0625] 3. Real-time sentiment analysis during interviews:

[0626] The server analyzes the candidate's video and audio data collected through the video chat system to assess their emotional state in real time. It measures the candidate's level of nervousness and confidence and presents specific questions to the interviewer. For example, it presents questions such as, "Tell us about a security incident you were actually involved in in the past."

[0627] 4. Post-interview scoring and comparative evaluation:

[0628] After the interview, the server integrates the data collected and uses an AI model to quantify and evaluate the candidate's match. A final score is calculated that takes into account emotional data, and multiple candidates are compared and evaluated. For example, Candidate A is scored 90 points, and Candidate B is scored 85 points.

[0629] Real-time analysis using an emotion engine and advanced analysis using generative AI models is expected to streamline the recruitment of talent for security departments and significantly improve corporate recruitment processes.

[0630] Example prompt sentence:

[0631] Analyze and show the candidate's emotional state in real time: Analyze the candidate's nervousness and confidence level. Evaluate the candidate's emotional state in real time and suggest questions that will appear to the interviewer.

[0632] If the candidate seems nervous, suggest questions: for example, about past project experience.

[0633] This allows for the quick and accurate selection of personnel suitable for the security department.

[0634] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0635] Step 1:

[0636] A user logs in to a recruitment portal site. The user uses a terminal to input the company's recruitment requirements. An example of input data is "Network security specialist with at least three years of experience." The input data is sent to the server by the terminal.

[0637] Step 2:

[0638] The server uses a generative AI model to automatically generate a job posting based on the received hiring requirements data. The job posting includes requirements such as "CISSP certification" and "practical experience in network security." The generated job posting is sent from the server to the terminal, where it can be viewed by the user.

[0639] Step 3:

[0640] Candidates upload their resumes and work histories to a recruitment portal site. The terminal sends these document data to a server. An example of input data is "I have experience in security development in the financial industry."

[0641] Step 4:

[0642] The server inputs the received document data into a natural language processing (NLP) engine. The NLP engine analyzes the document and extracts skill and experience information. For example, keywords such as "CISSP certification" and "network security experience" are extracted. The extracted information is stored on the server.

[0643] Step 5:

[0644] The server uses a generative AI model to evaluate the candidate's match based on the extracted skills and experience information. The evaluation results are sent from the server to the device in the form of "high match" or "medium match," and can be viewed by the user.

[0645] Step 6:

[0646] At the start of the interview, the user (interviewer) talks with the candidate through a video call system. The terminal transmits video and audio data to the server in real time. For example, video frames and audio streams are input.

[0647] Step 7:

[0648] The server uses a video analysis engine and an audio analysis engine to analyze the video and audio data in real time. The candidate's facial expressions and tone of voice are extracted from the analysis results. The emotion engine uses this to evaluate the candidate's level of nervousness and confidence. The evaluation results are sent from the server to the terminal in real time, where the interviewer can check them.

[0649] Step 8:

[0650] During the interview, the server generates candidate questions and concerns based on the results of sentiment analysis in real time and displays them on the device. For example, if the candidate is feeling nervous, the server will suggest a question such as, "Tell us about your specific project experience."

[0651] Step 9:

[0652] After the interview, the server combines the collected video and audio data and the interviewer's evaluation. This data is input into an AI model, which scores the candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[0653] Step 10:

[0654] The server adjusts the scoring results based on the emotion data obtained by the emotion engine. After calculating the final score, a comparative evaluation is performed based on multiple candidates to select the most suitable candidate. The selection results are sent from the server to the terminal in report format, allowing the user (company recruiter) to make the final decision.

[0655] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0656] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0657] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0658] [Second embodiment]

[0659] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0660] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0661] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0662] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0663] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0664] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0665] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0666] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0667] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0668] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0669] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0670] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0671] System Overview

[0672] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatically creating job postings, analyzing candidates' characteristics and determining their match, analyzing real-time emotions during interviews, and scoring and comparative evaluation after interviews.

[0673] Program processing flow and specific examples

[0674] Automatic creation of job postings

[0675] 1. A user (company representative) logs in to a recruitment portal site and enters the recruitment requirements. For example, suppose the company is looking for a "software development engineer with at least three years of experience."

[0676] 2. The terminal sends the entered employment requirements to the server.

[0677] 3. The server inputs the received hiring requirements into the AI ​​model, which then generates the optimal job posting for the target and position based on past job postings and hiring history.

[0678] 4. The server sends the generated job posting to the terminal so that the user can check it. After checking, the user can make any necessary corrections and officially publish it.

[0679] Candidate characteristics analysis and match assessment

[0680] 1. The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[0681] 2. The terminal sends the uploaded document to the server.

[0682] 3. The server inputs the received document into a natural language processing (NLP) engine. The NLP engine analyzes the document and extracts skill and experience information. Examples of extracted skills include "financial industry" and "software development experience."

[0683] 4. The server uses the extracted information to evaluate the candidate's match using an AI model. The evaluation results are provided to the user (company representative), who determines whether the candidate has a high match.

[0684] Real-time analysis during the interview

[0685] 1. The user (interviewer) starts the interview and uses the video call system.

[0686] 2. The device transmits the interview video and audio data to the server in real time.

[0687] 3. The server uses a video analysis engine to analyze the candidate's emotions based on their facial expressions and tone of voice, for example, to determine their level of nervousness or confidence.

[0688] 4. The server displays the sentiment analysis results in real time on the interviewer's device and suggests suitable questions and areas of concern. For example, if the candidate is nervous, the server will prompt them to "Tell me more about your specific project experience."

[0689] Post-interview scoring and comparison

[0690] 1. After the interview is completed, the server integrates the collected video and audio data and the interviewer's evaluation.

[0691] 2. The server uses an AI model to score each candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[0692] 3. The server compares candidates based on the scoring results and determines the best candidate.

[0693] 4. The server generates a report containing the comparison results and sends it to the terminal. The user (company representative) makes the final hiring decision based on this report.

[0694] Technical effects

[0695] This system allows companies to streamline their recruitment processes and significantly reduce the burden on managers. In addition, AI-based data analysis enables objective and accurate candidate evaluation, enabling the rapid recruitment of suitable personnel. This is expected to improve a company's competitiveness and optimize the allocation of personnel.

[0696] The processing flow will be explained below.

[0697] Automatic creation of job postings

[0698] Step 1:

[0699] The user (company representative) logs in to the recruitment portal site and enters the recruitment requirements, for example, "Recruiting a software development engineer with at least three years of experience."

[0700] Step 2:

[0701] The terminal transmits the entered employment requirements to the server.

[0702] Step 3:

[0703] The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[0704] Step 4:

[0705] The AI ​​model generates optimal job descriptions, including requirements such as "Java or Python programming experience" and "team development experience."

[0706] Step 5:

[0707] The server sends the generated job posting to the terminal so that the user can check it.

[0708] Step 6:

[0709] After checking the job posting, the user makes any necessary corrections, makes a final check, and then publishes the job posting.

[0710] ---

[0711] Candidate characteristics analysis and match assessment

[0712] Step 1:

[0713] The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[0714] Step 2:

[0715] The terminal transmits the uploaded document to the server.

[0716] Step 3:

[0717] The server inputs the received document into a natural language processing (NLP) engine.

[0718] Step 4:

[0719] The NLP engine analyzes documents and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[0720] Step 5:

[0721] Based on the extracted information, the server uses an AI model to evaluate the candidate's match, for example, rating them as a "high match."

[0722] Step 6:

[0723] The server sends the evaluation results to the terminal so that the user (company representative) can check them.

[0724] ---

[0725] Real-time analysis during the interview

[0726] Step 1:

[0727] The user (interviewer) starts the interview and uses the video call system.

[0728] Step 2:

[0729] The terminal transmits the interview video and audio data to the server in real time.

[0730] Step 3:

[0731] The server uses a video analysis engine and an audio analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[0732] Step 4:

[0733] Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Please tell us more about your specific project experience."

[0734] Step 5:

[0735] The user (interviewer) conducts the interview based on the presented questions and asks additional questions if necessary.

[0736] ---

[0737] Post-interview scoring and comparison

[0738] Step 1:

[0739] The server integrates the collected video and audio data and the interviewer's evaluation after the interview is completed.

[0740] Step 2:

[0741] The server uses an AI model to score each candidate's match, for example, Candidate A might get 90 points, and Candidate B might get 85 points.

[0742] Step 3:

[0743] The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[0744] Step 4:

[0745] The server generates a report containing the comparison results and sends it to the terminal.

[0746] Step 5:

[0747] The user (company representative) makes the final hiring decision based on the received report.

[0748] ---

[0749] Through the above processing steps, the recruitment assistant AI system realizes an efficient and objective recruitment process, significantly reducing the burden on corporate managers.

[0750] Example 1

[0751] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0752] Traditional corporate recruitment processes require a wide range of manual tasks, such as creating job postings, evaluating candidates, conducting interviews, and scoring the results, which is time-consuming and labor-intensive. It is also difficult to analyze candidates' characteristics and emotions in real time, making objective evaluations difficult. Therefore, there is a need to reduce the burden on recruiters and achieve a more efficient and objective recruitment process.

[0753] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0754] In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents and extracting skills and experience information using a natural language processing engine, means for evaluating the candidate's match based on the skills and experience information, means for analyzing video and audio during the interview in real time and displaying the results of a sentiment analysis, means for presenting suitable question candidates and concerns during the interview, means for scoring the candidate's match after the interview, and means for comparatively evaluating multiple candidates based on the scoring results, thereby enabling efficiency improvement and objective evaluation of the entire hiring process.

[0755] "Employment requirements" refer to the specific conditions, such as skills, experience, educational background, and qualifications, that a company seeks in its employees.

[0756] A "job posting" is a document containing recruitment information published by a company, which lists the job types, conditions, and work content.

[0757] "Documents" refers to documents submitted by candidates, such as resumes and application forms.

[0758] A "natural language processing engine" is software or a system that analyzes the content of documents and extracts skill and experience information.

[0759] "Skills and experience information" is information that indicates the candidate's abilities and past work experience.

[0760] "Match" is an evaluation that indicates how closely a candidate's characteristics match the company's hiring requirements.

[0761] "Real-time analysis of video and audio" refers to a technology that analyzes video and audio collected during an interview on the spot.

[0762] "Emotion analysis results" are data showing the emotional state of a candidate obtained by analyzing their facial expressions and tone of voice.

[0763] "Suggested Questions and Concerns" present questions that interviewers should ask candidates and points to be aware of during the interview.

[0764] "Scoring" refers to the process of assigning a score to the interview results and the candidate's characteristics to evaluate them.

[0765] "Comparative evaluation" means comparing multiple candidates based on scoring results.

[0766] System Overview

[0767] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatically creating job postings, analyzing candidates' characteristics and determining their match, analyzing real-time emotions during interviews, and scoring and comparative evaluation after interviews.

[0768] Automatic creation of job postings

[0769] The user (company representative) accesses the login page using their company's account information and enters specific hiring requirements, such as "software development engineer with at least three years of experience." The device collects the information entered by the user in real time and sends it securely to the server using HTTPS. The server converts the received hiring requirements into an appropriate format and inputs it into an AI model using TensorFlow. The model references a past database and generates an optimal job posting. The server sends the generated job posting in JSON format to the device, which displays it on the user's screen. The user checks the content on the screen, modifies the job posting if necessary, and clicks the "Publish" button, which officially posts the job information on the portal site.

[0770] Examples:

[0771] If a user enters a job requirement for "an engineer with 5+ years of experience in front-end development," the server generates a corresponding job posting and provides it as "React.js developer with 5+ years of experience wanted." The user can then review the generated job posting, modify it, and publish it.

[0772] Example prompt:

[0773] "Please create a job posting for an engineer with at least 5 years of experience in front-end development."

[0774] Candidate characteristics analysis and match assessment

[0775] Users (candidates) upload their resumes and application forms to the system in an appropriate format (PDF or Word). The terminal scans the uploaded files and sends them to the server. The server launches a natural language processing (NLP) engine using PyTorch to analyze the text within the document. For example, it extracts keywords such as "financial industry" and "software development experience." The server inputs the extracted information into a pre-prepared evaluation algorithm and numerically evaluates the match between the candidate's skills and the company's requirements. The evaluation results are sent to the terminal, and the user can view a detailed evaluation of the candidate on the screen.

[0776] Examples:

[0777] If a candidate states that they have "software development experience in the financial industry" in the past, the server will extract "financial industry" and "software development experience" and compare them with the company's job requirements to evaluate the match as 90%.

[0778] Example prompt:

[0779] "Analyze the resumes of candidates with software development experience in the financial industry to determine the degree of match."

[0780] Real-time sentiment analysis during interviews

[0781] The user (interviewer) initiates a video call with a candidate via Zoom or Microsoft Teams. The device captures the video and audio data of the video call in real time and sends it to the server. The server runs a video analysis engine such as OpenCV to analyze the candidate's facial expressions and tone of voice. For example, it analyzes their level of nervousness and confidence. The server then creates candidate questions corresponding to the analysis results and displays them in real time on the interviewer's device. The interviewer can use this information to ask the candidate appropriate questions.

[0782] Examples:

[0783] If sentiment analysis reveals that a candidate is nervous during an interview, the server will display "High nervousness" and suggest questions to the interviewer, such as "Tell me more about your specific project experience."

[0784] Example prompt:

[0785] "Perform real-time sentiment analysis of candidate tension during interviews and suggest appropriate questions."

[0786] Post-interview scoring and comparison

[0787] The server combines the video and audio data collected during the interviews, as well as the interviewers' manual evaluations, and stores them as a single dataset. The server inputs the combined dataset into an AI model, which evaluates each candidate's skills, aptitude, and expressions during the interview to create a score. For example, candidate A may receive a score of 90, while candidate B receives a score of 85. The server compares the scoring results and determines the most suitable candidate, taking into account algorithms and past hiring performance data. The server generates a detailed report including the final evaluation and comparison results and sends it to the device. The user makes the final hiring decision based on this report. The user reviews the report displayed on the device and selects the most suitable candidate. Once the final decision is made, the server sends an offer of employment.

[0788] Technical effects

[0789] This system allows companies to streamline their recruitment processes and significantly reduce the burden on managers. In addition, AI-based data analysis enables objective and accurate candidate evaluation, enabling the rapid recruitment of suitable personnel. This is expected to improve a company's competitiveness and optimize the allocation of personnel.

[0790] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0791] Step 1:

[0792] A user logs in to a recruitment portal site and enters their recruitment requirements.

[0793] Input: User inputs recruitment requirements (e.g., "Software development engineer with at least 3 years of experience")

[0794] What happens: The user logs in using their company's account information and enters their job requirements in the text fields.

[0795] Step 2:

[0796] The terminal transmits the entered employment requirements to the server.

[0797] Input: Recruitment requirements data

[0798] Data processing / calculation: The device collects input information in real time and sends it securely to the server using HTTPS

[0799] Output: Recruitment requirements data sent to the server

[0800] Specific operation: The terminal registers data in the transmission queue and sends it to the server via HTTPS communication.

[0801] Step 3:

[0802] The server inputs the recruitment requirements received into the AI ​​model and generates a job posting.

[0803] Input: Recruitment requirements data received by the server

[0804] Data processing / calculation: The server converts the data into the appropriate format, inputs it into an AI model using TensorFlow, and generates a job posting.

[0805] Output: Generated job posting data

[0806] Specific operation: The server retrieves relevant information from the database and inputs it into the AI ​​model to generate a job posting.

[0807] Step 4:

[0808] The server sends the generated job posting to the terminal and the user confirms it.

[0809] Input: Generated job posting data

[0810] Data processing / calculation: The server converts the generated job posting into JSON format and sends it to the terminal.

[0811] Output: The job posting displayed to the user

[0812] Specific operation: The server sends JSON format data to the terminal, which parses it and displays it on the user's screen.

[0813] Step 5:

[0814] The user modifies and publishes the job posting.

[0815] Input: User-modified job posting data

[0816] Data processing / calculation: The user edits the job posting details on the screen and clicks the "Publish" button.

[0817] Output: Published job postings

[0818] Specific operation: The user checks the content, makes corrections, and then presses the publish button, and the job information is posted on the portal site.

[0819] Step 6:

[0820] The user (candidate) uploads their resume and application form.

[0821] Input: Candidates upload their resumes and application forms

[0822] Data processing / calculation: Upload document data to the system

[0823] Output: Uploaded document data

[0824] What happens: Candidates follow the system instructions and upload documents using the file selection button.

[0825] Step 7:

[0826] The terminal sends the uploaded document to the server.

[0827] Input: Uploaded document data

[0828] Data processing / calculation: The device scans the file and sends it to the server

[0829] Output: Document data sent to the server

[0830] Specific operation: The device temporarily stores the uploaded file and securely sends it to the server.

[0831] Step 8:

[0832] The server inputs the document into an NLP engine for analysis.

[0833] Input: Document data sent to the server

[0834] Data processing / calculation: The server runs a natural language processing (NLP) engine using PyTorch to perform analysis. Specifically, it extracts keywords such as "financial industry" and "software development experience."

[0835] Output: Parsed skills and experience information

[0836] How it works: The server inputs documents into the NLP engine and extracts important skills and experience information.

[0837] Step 9:

[0838] The server uses the analysis results to evaluate the degree of match using an AI model.

[0839] Input: Parsed skills and experience information

[0840] Data processing / calculation: The server inputs the extracted information into an evaluation algorithm and evaluates the candidate's match numerically.

[0841] Output: Evaluation results

[0842] Specific operation: The server scores the extracted information based on an evaluation algorithm.

[0843] Step 10:

[0844] The server provides the evaluation results to the user.

[0845] Input: Evaluation result data

[0846] Data processing / calculation: Evaluation results sent to the terminal

[0847] Output: Evaluation results displayed on the user's screen

[0848] Specific operation: The server sends the evaluation results in JSON format to the terminal, and the user confirms them.

[0849] Step 11:

[0850] The user (interviewer) starts the interview using the video call system.

[0851] Input: Start the video call system

[0852] Data processing / calculation: Capture of interview video and audio data

[0853] Output: Captured video and audio data

[0854] What happens: The interviewer will start a video call using Zoom or Microsoft Teams.

[0855] Step 12:

[0856] The terminal transmits the interview video and audio data to the server.

[0857] Input: Captured video and audio data

[0858] Data processing / calculation: Send to server in real time

[0859] Output: Video and audio data sent to the server

[0860] Specific operation: The device captures video and audio data in real time and sends it to the server.

[0861] Step 13:

[0862] The server performs emotion analysis using a video analysis engine.

[0863] Input: Video and audio data sent to the server

[0864] Data processing / calculation: Emotion analysis is performed using OpenCV etc. to analyze tension and confidence.

[0865] Output: Sentiment analysis results

[0866] Specific operation: The server inputs video and audio data into an analysis engine to evaluate the emotional state.

[0867] Step 14:

[0868] The server displays the analysis results and suitable question candidates on the device.

[0869] Input: Sentiment analysis results

[0870] Data processing / calculation: Generate suitable question candidates and send them to the device

[0871] Output: Question candidates and sentiment analysis results displayed on the user's screen

[0872] Specific operation: The server generates candidate questions based on the analysis results and sends them to the device, which then displays them to the user.

[0873] Step 15:

[0874] The server consolidates the collected data.

[0875] Input: Video and audio data collected during the interview, and manual assessment by the interviewer

[0876] Data processing / calculation: Data integration and storage

[0877] Output: A consolidated dataset

[0878] Specific operation: The server consolidates and centrally manages all data.

[0879] Step 16:

[0880] The server scores each candidate using an AI model.

[0881] Input: Integrated dataset

[0882] Data processing / calculation: Input data into the AI ​​model to generate a score for each candidate (e.g., candidate A scores 90, candidate B scores 85)

[0883] Output: Scoring results

[0884] How it works: The server inputs data into the AI ​​model and generates a score.

[0885] Step 17:

[0886] The server compares the candidates to determine the best candidate.

[0887] Input: Scoring results

[0888] Data processing / calculation: Use comparison algorithms to determine the best candidates

[0889] Output: The best candidate is identified

[0890] Specific operation: The server compares the scoring results and selects the most suitable candidate.

[0891] Step 18:

[0892] The server sends the comparison results as a report to the terminal.

[0893] Input: The best candidate's results

[0894] Data processing / calculation: Generate detailed reports and send them to your device

[0895] Output: The report that is displayed on the user's screen

[0896] Specific operation: The server generates a report based on the evaluation results and sends it to the terminal, which then displays it to the user.

[0897] Step 19:

[0898] The user makes the final hiring decision based on the report.

[0899] Input: Best candidate report

[0900] Data processing / calculation: Making decisions based on reports

[0901] Output: Final hiring decision

[0902] Specific actions: The user reviews the report, selects the best candidates, and sends out job offers.

[0903] (Application example 1)

[0904] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0905] In the recruitment process, analyzing candidate characteristics and emotions during interviews is important, but there is no system that utilizes this data in real time to evaluate risk factors. As a result, companies are forced to rely on subjective judgment when selecting appropriate candidates, making efficient recruitment difficult. In addition, it is difficult to detect security risks early in post-recruitment activities.

[0906] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0907] In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and displaying the emotion analysis results, means for analyzing the candidate's behavior in real time during the interview and performing a risk assessment, means for scoring the candidate's match after the interview, and means for comparatively assessing multiple candidates based on the scoring results. This enables a more objective and efficient hiring process based on real-time emotion and risk analysis.

[0908] "Means for inputting a company's recruitment requirements" refers to an interface that allows a company to input the conditions, qualifications, skills, etc. of the personnel it wishes to hire into the system.

[0909] The "means for automatically generating a job posting based on the hiring requirements" refers to an algorithm and system that creates optimal job advertisements and job postings based on the hiring requirements entered.

[0910] "Means for receiving documents submitted by candidates" refers to a function for receiving resumes and application forms uploaded by candidates to the system.

[0911] The "means for analyzing the document and extracting skill and experience information" is a function that uses natural language processing technology to extract information such as the candidate's skills and work history from the received document.

[0912] The "means for evaluating the degree of match of a candidate based on the skills and experience information" refers to an algorithm and system that compares the extracted skills and experience information and evaluates the degree of suitability of a candidate for the recruitment requirements.

[0913] "A means of analyzing video and audio during interviews in real time and displaying the results of emotional analysis" is a function that instantly processes video and audio collected during interviews, analyzes the candidate's facial expressions and tone of voice, and displays their emotional state.

[0914] "Means for analyzing candidate behavior in real time during interviews and conducting risk assessments" refers to a function that observes candidate actions and behaviors occurring during interviews in real time and assesses risk based on that information.

[0915] "Means for scoring candidate match after interview" refers to algorithms and systems that quantify and evaluate a candidate's suitability based on data collected after the interview.

[0916] The "means for comparatively evaluating a plurality of candidates based on the scoring results" is a function for comparing the evaluation results of scored candidates and selecting the most suitable candidate.

[0917] This invention provides a recruitment assistant AI system that integrates multiple functions to streamline a company's recruitment process and reduce the burden on managers. The configuration and operation of this system are described below.

[0918] Automatic creation of job postings

[0919] The server automatically generates job postings based on information received through the company's recruitment requirements input method. At this time, the server uses a generative AI model based on past recruitment data and recruitment performance to create the optimal job posting. The user can check this job posting via their device, make any necessary corrections, and publish the job information.

[0920] Candidate characteristics analysis and match assessment

[0921] The server receives the resumes and application forms submitted by candidates. Based on this, it analyzes the documents using natural language processing (NLP) technology to extract skill and experience information. The server then inputs the extracted information into a generative AI model to evaluate the candidate's match.

[0922] Real-time sentiment analysis and risk assessment during interviews

[0923] The user (interviewer) uses smart glasses during the interview to collect video and audio data. The server analyzes this data in real time, analyzing the candidate's facial expressions and tone of voice to understand their emotional state. It also analyzes the candidate's behavior in real time during the interview and performs a risk assessment. This data is displayed on the interviewer's device and serves as a reference for how to respond.

[0924] Post-interview scoring and comparative evaluation

[0925] After the interviews are completed, the server integrates the collected data and scores the candidates' match. It uses a generative AI model to evaluate each candidate's suitability and compare multiple candidates. It then generates a report of the best candidates and provides it to the recruiter.

[0926] Hardware and software used

[0927] This system mainly uses the following hardware and software:

[0928] Hardware: Smart glasses, devices (PC, tablet, etc.)

[0929] Software: Python, OpenCV, Keras, NLP engine, generative AI model

[0930] Examples of specific examples and prompts

[0931] For example, if a company is looking for a "software development engineer with at least three years of experience," the server will generate the most suitable job posting based on this information and analyze the candidate's submitted documents. During the interview, the smart glasses will analyze the candidate's behavior in real time and perform a risk assessment.

[0932] Example prompt sentence:

[0933] "Develop an application that uses camera footage to analyze candidates' facial expressions in real time, determine their emotions, and conduct a risk assessment. Display the results in real time."

[0934] In this way, the present invention assists in the entire recruitment process, allowing companies to efficiently recruit the right talent.

[0935] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0936] Step 1:

[0937] The user (company representative) enters the recruitment requirements.

[0938] Input: Job requirements (e.g., "Software development engineer with at least 3 years of experience")

[0939] Output: The entered recruitment requirements are sent to the server.

[0940] Specific operations: The company representative logs in to the recruitment portal site and enters the necessary recruitment requirements in detail into the form.

[0941] Step 2:

[0942] The server automatically generates a job posting based on the received hiring requirements.

[0943] Input: Recruitment requirements

[0944] Output: Generated job posting

[0945] Specific operation: The server uses the recruitment requirement data input into the generative AI model, refers to past cases, and creates the most appropriate job posting.

[0946] Step 3:

[0947] The server sends the generated job posting to the user's (company representative's) terminal.

[0948] Input: Generated job posting

[0949] Output: Job posting confirmation screen

[0950] Specific operation: The server sends the created job posting to the company's employee's terminal and displays a screen where the employee can check and edit the job posting.

[0951] Step 4:

[0952] The user uploads the candidate's documents (resume and application form).

[0953] Input: resume and application form

[0954] Output: Uploaded documents are sent to the server.

[0955] What happens: Candidates use a web portal to upload their resume and application form.

[0956] Step 5:

[0957] The server analyzes the uploaded documents using a natural language processing (NLP) engine.

[0958] Input: resume and application form

[0959] Output: Extraction results of skills and experience information

[0960] Specific operation: The server uses an NLP engine to analyze the contents of the document and extract skill and experience information such as "software development experience in the financial industry."

[0961] Step 6:

[0962] The server evaluates the match of the candidate based on the extracted information.

[0963] Input: Extracted skills and experience information

[0964] Output: Candidate match rating

[0965] How it works: The server uses a generative AI model to match the extracted information with the job requirements and score the match.

[0966] Step 7:

[0967] The user (interviewer) uses smart glasses during the interview to collect video and audio of the candidate in real time.

[0968] Input: Video and audio data from the interview

[0969] Output: Real-time data sent to the server

[0970] How it works: The interviewer puts on the smart glasses and begins the interview. The glasses' camera and microphone collect video and audio of the candidate in real time.

[0971] Step 8:

[0972] The server analyzes the data received in real time and displays an assessment of emotional state and behavioral risk.

[0973] Input: Video and audio data from the interview

[0974] Output: Real-time display of emotional state and risk assessment

[0975] Specific operation: The server inputs the received data into an analysis engine, analyzes the candidate's emotional state (e.g., level of tension, confidence), and displays the evaluation results on the interviewer's terminal.

[0976] Step 9:

[0977] The server consolidates the data collected after the interview and scores the candidate's match.

[0978] Input: Video and audio data collected during the interview, and interviewer evaluation

[0979] Output: Match score

[0980] How it works: The server consolidates the collected data and uses a generative AI model to score candidates for match.

[0981] Step 10:

[0982] The server performs comparative evaluation based on the scoring results of multiple candidates and generates a report.

[0983] Input: Match score for each candidate

[0984] Output: Comparative evaluation report

[0985] Specific operation: The server compares the scoring results of each candidate, generates a report to select the most suitable candidate, and provides it to the recruiter.

[0986] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0987] System Overview

[0988] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. In particular, by incorporating an emotion engine, it enables more precise analysis that takes into account the candidate's emotional state during and after the interview. This system supports each stage of the recruitment process, including automatic creation of job postings, analysis of candidate characteristics and match determination, real-time emotion analysis using the emotion engine, and post-interview scoring and comparative evaluation.

[0989] Program processing flow and specific examples

[0990] Automatic creation of job postings

[0991] 1. A user (company representative) logs in to a recruitment portal site and enters the recruitment requirements. For example, they might enter "Recruiting a software development engineer with at least three years of experience."

[0992] 2. The terminal sends the entered employment requirements to the server.

[0993] 3. The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[0994] 4. The AI ​​model generates the most suitable job description, including requirements such as "Java and Python programming experience" and "team development experience."

[0995] 5. The server sends the generated job posting to the terminal so that the user can view it.

[0996] 6. After checking the job posting, the user will make any necessary corrections, make a final confirmation, and then publish it.

[0997] Candidate characteristics analysis and match assessment

[0998] 1. The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience developing software in the financial industry."

[0999] 2. The terminal sends the uploaded document to the server.

[1000] 3. The server inputs the received document into a natural language processing (NLP) engine.

[1001] 4. The NLP engine analyzes the document and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[1002] 5. Based on the extracted information, the server uses an AI model to evaluate the candidate's match, e.g., a "high match."

[1003] 6. The server sends the evaluation results to the terminal so that the user (company representative) can check them.

[1004] Real-time analysis during the interview

[1005] 1. The user (interviewer) starts the interview and uses the video call system.

[1006] 2. The device transmits the interview video and audio data to the server in real time.

[1007] 3. The server uses a video analysis engine and a voice analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[1008] 4. The server uses an emotion engine to perform a more detailed analysis of the candidate's emotional state, assessing their level of nervousness, confidence, etc.

[1009] 5. Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Tell me more about your specific project experience."

[1010] 6. The user (interviewer) conducts the interview based on the questions presented and asks additional questions if necessary.

[1011] Post-interview scoring and comparison

[1012] 1. The server integrates the video and audio data collected after the interview and the interviewer's evaluation.

[1013] 2. The server uses an AI model to score each candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[1014] 3. The server adjusts the scoring results based on the emotion data obtained by the emotion engine, and calculates the final score taking into account the level of tension and confidence from the emotion data.

[1015] 4. The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[1016] 5. The server generates a report containing the comparison results and sends it to the terminal.

[1017] 6. The user (company representative) makes the final hiring decision based on the received report.

[1018] Technical effects

[1019] This system enables companies to streamline their recruitment processes and significantly reduce the burden on managers. In particular, by analyzing emotional data using an emotion engine, it is possible to grasp the true state of candidates during interviews, enabling more accurate evaluations. This will enable companies to quickly hire the right talent, which is expected to improve a company's competitiveness and optimize personnel allocation.

[1020] The processing flow will be explained below.

[1021] Automatic creation of job postings

[1022] Step 1:

[1023] The user (company representative) logs in to the recruitment portal site and enters the recruitment requirements, for example, "Recruiting a software development engineer with at least three years of experience."

[1024] Step 2:

[1025] The terminal transmits the entered employment requirements to the server.

[1026] Step 3:

[1027] The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[1028] Step 4:

[1029] The AI ​​model generates optimal job postings, such as those that include requirements such as "Java or Python programming experience" and "team development experience."

[1030] Step 5:

[1031] The server sends the generated job posting to the terminal, where it is displayed so that the user can check it.

[1032] Step 6:

[1033] The user checks the generated job posting, modifies it as necessary, and then publishes it.

[1034] ---

[1035] Candidate characteristics analysis and match assessment

[1036] Step 1:

[1037] Users (candidates) upload their resumes and application forms to a recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[1038] Step 2:

[1039] The terminal transmits the uploaded document to the server.

[1040] Step 3:

[1041] The server inputs the received document into a natural language processing (NLP) engine.

[1042] Step 4:

[1043] The NLP engine analyzes documents and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[1044] Step 5:

[1045] Based on the extracted information, the server uses an AI model to evaluate the candidate's match, for example, rating them as a "high match."

[1046] Step 6:

[1047] The server sends the evaluation results to the terminal and displays them so that the user (company representative) can check them.

[1048] ---

[1049] Real-time analysis during the interview

[1050] Step 1:

[1051] The user (interviewer) starts the interview and uses the video call system.

[1052] Step 2:

[1053] The terminal transmits the interview video and audio data to the server in real time.

[1054] Step 3:

[1055] The server uses a video analysis engine and an audio analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[1056] Step 4:

[1057] The server uses an emotion engine to further analyze the candidate's emotional state, assessing, for example, their level of nervousness and confidence.

[1058] Step 5:

[1059] Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Please tell us more about your specific project experience."

[1060] Step 6:

[1061] The user (interviewer) conducts the interview based on the presented questions and asks additional questions as necessary.

[1062] ---

[1063] Post-interview scoring and comparison

[1064] Step 1:

[1065] The server integrates the collected video and audio data and the interviewer's evaluation after the interview is completed.

[1066] Step 2:

[1067] The server uses an AI model to score each candidate's match, for example, Candidate A might get 90 points, and Candidate B might get 85 points.

[1068] Step 3:

[1069] The server adjusts the scoring results based on the emotion data obtained by the emotion engine, and calculates a final score that takes into account the level of tension and confidence from the emotion data.

[1070] Step 4:

[1071] The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[1072] Step 5:

[1073] The server generates a report containing the comparison results and sends it to the terminal.

[1074] Step 6:

[1075] The user (company representative) makes the final hiring decision based on the received report.

[1076] ---

[1077] Through the above processing steps, this system streamlines the entire recruitment process, reducing the burden on managers, and improves the accuracy of candidate aptitude assessment through detailed emotion analysis using an emotion engine.

[1078] Example 2

[1079] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1080] Traditional recruitment processes are time-consuming and labor-intensive, and rely on human evaluation, making them difficult to operate efficiently, especially when there are a large number of applicants. It is also difficult to determine a candidate's actual emotional state during the interview, making evaluations subjective. This increases the risk of delays in hiring the right talent and increases the likelihood of mismatches. Therefore, there is a need for a way to streamline the entire recruitment process and improve accuracy.

[1081] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1082] In this invention, the server includes means for inputting hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and providing emotion analysis results, means for scoring the candidate's match after the interview, means for comparatively evaluating multiple candidates based on the scoring results, means for adjusting the scoring results using emotion data, and means for selecting the most suitable candidate based on the adjusted scoring results. This makes the entire hiring process more efficient and enables more accurate and objective evaluations.

[1083] The "means for inputting recruitment requirements" is a method for providing an interface for users to input the conditions and needs of a company regarding recruitment.

[1084] "Means for automatically generating a job posting based on the hiring requirements" refers to a method for automatically creating an appropriate job posting using an AI model based on the hiring requirements entered.

[1085] The "means for receiving documents submitted by candidates" refers to a method for receiving documents such as resumes and application forms uploaded by candidates onto a server.

[1086] The "means for analyzing the document and extracting skill and experience information" is a method for analyzing the received document using natural language processing technology and extracting information such as the candidate's skills and work experience.

[1087] "Means for evaluating the degree of match of a candidate based on the skills and experience information" refers to a method for evaluating the degree to which a candidate matches the company's recruitment requirements based on the extracted skills and experience information.

[1088] "Means for analyzing video and audio data during an interview in real time and providing emotional analysis results" refers to a method for analyzing video and audio data acquired during an interview in real time, determining the emotional state of the candidate, and providing the results to the interviewer.

[1089] "Method of scoring candidate match after interview" is a method of scoring each candidate's suitability based on interview data.

[1090] The "means for comparatively evaluating a plurality of candidates based on the scoring results" refers to a method for comparing a plurality of candidates based on the scoring results and selecting the candidate with the highest degree of suitability.

[1091] "Means for adjusting scoring results using emotional data" refers to a method for correcting scoring results based on emotional data obtained during the interview and evaluating the final degree of fit.

[1092] The "means for selecting the most suitable candidate based on the adjusted scoring results" is a method for selecting the most suitable candidate based on the adjusted scoring results.

[1093] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatic creation of job postings, candidate characteristic analysis and match assessment, real-time emotion analysis using an emotion engine, and post-interview scoring and comparative evaluation. In particular, the incorporation of an emotion engine enables more precise analysis that takes into account the candidate's emotional state during and after the interview.

[1094] Hardware and software used

[1095] The server will be a high-performance cloud server. Specifically, cloud platforms such as Amazon Web Services (AWS) and Microsoft Azure are considered. Standard PCs, tablets, and smartphones can be used as terminals. The following software will be used:

[1096] Natural language processing model (NLP engine): SpaCy or Stanford NLP

[1097] Generative AI models: GPT-3 and BERT

[1098] Video analysis engine: OpenCV

[1099] Voice analysis engine: Librosa

[1100] Sentiment analysis engine: AI model equipped with FaceAPI and Sentiment Analysis

[1101] Video calling system: Zoom or Microsoft Teams

[1102] Program processing

[1103] Automatic creation of job postings

[1104] The user (company representative) logs in to the recruitment portal site and enters their hiring requirements. For example, they may enter requirements such as "We are looking for a software development engineer with at least three years of experience." The device then sends the entered data to the server. The server then inputs the received data into a generative AI model, compares it with past job posting data, and generates the most suitable job posting. The generated job posting is then sent to the device, where it can be viewed by the user. Any necessary corrections are made, and the job posting is published after a final confirmation.

[1105] Candidate characteristics analysis and match assessment

[1106] The user (candidate) uploads their resume and application form to a recruitment portal site. For example, they may upload a document stating that they have previous experience developing software in the financial industry. The device then sends the uploaded document to the server. The server inputs the received document into an NLP engine, analyzes the document, and extracts skill and experience information. Based on the extracted information, an AI model is used to evaluate the candidate's match. The evaluation results are sent to the device and can be viewed by the user.

[1107] Real-time analysis during the interview

[1108] The user (interviewer) begins the interview using a video call system. The device sends the interview video and audio data to the server in real time. The server uses a video analysis engine and an audio analysis engine to analyze the video and audio data in real time. The emotion analysis engine is used to perform a detailed analysis of the candidate's emotional state and evaluate their level of nervousness and confidence. The analysis results are displayed on the interviewer's device as suitable question suggestions and areas of concern. For example, if the candidate is nervous, a question such as "Tell me more about your specific project experience" will be presented.

[1109] Post-interview scoring and comparative evaluation

[1110] Once the interview is over, the server combines the collected video and audio data and the interviewer's evaluation. It then uses an AI model to score each candidate's match. For example, candidate A might be rated 90 points, and candidate B 85 points. The scoring results are then adjusted based on emotional data to calculate the final score. Multiple candidates are compared and evaluated to determine the most suitable candidate. A report containing the final comparison results is generated and sent to the device. The user (company representative) makes the final hiring decision based on this report.

[1111] Examples of concrete examples and prompts

[1112] For example, use the following prompt:

[1113] 1. "We are looking for a software development engineer with at least 3 years of experience. Experience with Java or Python programming and team development is essential."

[1114] 2. "I have previous software development experience in the financial industry."

[1115] 3. "Tell me more about your specific project experience."

[1116] This will enable companies to streamline the recruitment process and significantly reduce the burden on managers. In particular, analyzing emotional data using an emotion engine will enable candidates to understand their true state during interviews, enabling more accurate evaluations.

[1117] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1118] Step 1:

[1119] A user logs in to a recruiting portal site.

[1120] Specific operation: The user accesses the login screen of the portal site using their company account and enters their authentication information. The server verifies the entered authentication information and authenticates the user. If authentication is successful, the user can access the main screen of the recruitment portal site.

[1121] Step 2:

[1122] The user enters the hiring requirements.

[1123] Input: Requirements such as job type, required skills, years of experience, etc.

[1124] Specific operation: The user enters specific employment requirements into a form on the site, such as "Seeking a software development engineer with at least three years of experience. Experience in Java or Python programming and team development is essential." The terminal then displays the entered information and allows the user to confirm it.

[1125] Step 3:

[1126] The terminal transmits the input data to the server.

[1127] Input: User-entered recruitment requirements

[1128] Output: Requirements data sent to the server

[1129] Specific operation: The terminal sends the recruitment requirement data to the server using an HTTP POST request. The server parses the received data and stores it in an appropriate format.

[1130] Step 4:

[1131] The server inputs the data into a generative AI model for analysis.

[1132] Input: Recruitment requirements data

[1133] Output: Generated job posting

[1134] How it works: The server inputs the received recruitment requirements data into a natural language processing model such as GPT-3 or BERT. The AI ​​model then references past recruitment data and generates a job posting that best fits the input requirements.

[1135] Step 5:

[1136] The server sends the generated job posting to the terminal.

[1137] Input: Generated job posting

[1138] Output: Job posting displayed on terminal

[1139] Specific operation: The server sends the generated job posting data to the terminal so that the user can check it. The terminal displays the received data and provides an interface that allows the user to check and modify it.

[1140] Step 6:

[1141] The user reviews and publishes the job posting.

[1142] Specific operation: The user checks the job posting displayed on the device and makes any necessary corrections. After completing the final check, the user presses the "Publish" button to publish the job posting. The device then sends the published information to the server and publishes the job posting.

[1143] Step 7:

[1144] The user uploads a resume or application form.

[1145] Input: resume and application form

[1146] Specific operation: The user (candidate) uses the upload screen on the recruitment portal site to select and upload the necessary documents such as resumes and application forms. The terminal then sends the selected documents to the server.

[1147] Step 8:

[1148] The terminal sends the document to the server.

[1149] Input: Uploaded document

[1150] Output: Document data stored on the server

[1151] Specific operation: The terminal sends the document data uploaded by the user to the server, and the server stores the received data in an appropriate format.

[1152] Step 9:

[1153] The server inputs the document into the NLP engine.

[1154] Input: Uploaded document data

[1155] Output: Skills and experience information

[1156] How it works: The server inputs the received document data into a natural language processing engine such as SpaCy or Stanford NLP for analysis. The NLP engine extracts keywords related to skills and work experience from the document.

[1157] Step 10:

[1158] The server evaluates the match of the candidate based on the extracted information.

[1159] Input: Skills and experience information

[1160] Output: Candidate match evaluation results

[1161] Specific operation: The server inputs the skills and experience information extracted by the NLP engine into the generative AI model to evaluate the candidate's match with the recruitment requirements. The evaluation result (e.g., "high match") is scored based on the company's recruitment criteria.

[1162] Step 11:

[1163] The server transmits the evaluation results to the terminal.

[1164] Input: Match evaluation result

[1165] Output: Evaluation results displayed on the terminal

[1166] Specific operation: The server sends the evaluation results to the terminal so that the user can check them. The terminal displays the received evaluation results and notifies the user.

[1167] Step 12:

[1168] The user starts the interview.

[1169] Specific operation: The user (interviewer) starts an interview with a candidate using a video call system such as Zoom or Microsoft Teams. The user accesses the video call system from their device and invites the candidate to participate.

[1170] Step 13:

[1171] The device transmits the interview video and audio data to the server in real time.

[1172] Input: Video and audio data from the interview

[1173] Output: Real-time data sent to the server

[1174] Specific operation: The device streams video and audio data captured during the interview to the server in real time, and the server prepares the infrastructure to process the received data appropriately.

[1175] Step 14:

[1176] The server analyzes the data and uses an emotion engine to analyze the emotional state.

[1177] Input: Real-time video and audio data

[1178] Output: Analysis of the candidate's emotional state

[1179] Specific operation: The server analyzes video and audio data using a video analysis engine (OpenCV) and an audio analysis engine (Librosa). The server inputs the analysis results into an emotion engine, which analyzes the candidate's facial expressions and tone of voice in real time.

[1180] Step 15:

[1181] The server sends potential questions and concerns to the device.

[1182] Input: Sentiment analysis results

[1183] Output: Possible questions and concerns displayed on the device

[1184] Specific operation: The server generates suitable candidate questions and concerns based on the results of sentiment analysis. The server then sends the generated information to the terminal in real time and presents it to the user (interviewer).

[1185] Step 16:

[1186] The user conducts the interview based on the questions presented.

[1187] Specific operation: The user conducts the interview while referring to the candidate questions and concerns displayed on the device. If necessary, the user asks additional questions and collects the candidate's answers.

[1188] Step 17:

[1189] The server consolidates the data after the interview is completed.

[1190] Input: Interview video, audio data, interviewer notes

[1191] Output: Consolidated interview data

[1192] Specific operation: After the interview is completed, the server integrates the collected video and audio data, as well as the interviewer's evaluation notes. The server then integrates and centrally manages this data.

[1193] Step 18:

[1194] The server uses an AI model to score each candidate's match.

[1195] Input: Consolidated interview data

[1196] Output: Match score for each candidate

[1197] Specific operation: The server inputs the integrated interview data into the generative AI model and scores each candidate's match. For example, candidate A might receive a score of 90, and candidate B might receive a score of 85.

[1198] Step 19:

[1199] The server adjusts the scoring results.

[1200] Input: Emotion data, scoring results

[1201] Output: Adjusted score

[1202] How it works: The server modifies the scoring results based on emotional data obtained during the interview, for example, adjusting the final score to take into account the level of nervousness or confidence.

[1203] Step 20:

[1204] The server compares and evaluates multiple candidates.

[1205] Input: Adjusted score

[1206] Output: Evaluation results for the best candidates

[1207] Specific operation: The server compares and evaluates multiple candidates based on the adjusted scores. The server selects the candidate with the highest fit.

[1208] Step 21:

[1209] The server generates a report and sends it to the device.

[1210] Input: Evaluation results of the best candidate

[1211] Output: Generated report

[1212] Specific operation: The server automatically generates a report summarizing the evaluation results in detail. The generated report is sent from the server to the terminal so that the user can check it.

[1213] Step 22:

[1214] The user makes the final hiring decision.

[1215] Specific operation: The user checks the report displayed on the terminal and makes a final hiring decision. If necessary, a final interview or additional checks are conducted, and then a formal hiring decision is issued.

[1216] This will streamline the entire hiring process and allow for more accurate and objective evaluations.

[1217] (Application example 2)

[1218] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1219] The recruitment process in a company is extremely important, especially in the security department, and requires accurate assessment of candidates' reliability and suitability. However, this process places a heavy burden on recruiters, making it difficult to secure the right talent. It is also difficult to grasp a candidate's emotional state in real time and make an accurate assessment. To solve these issues, a system is needed to streamline the entire recruitment process and assess candidates' reliability and nervousness.

[1220] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and displaying the emotion analysis results, means for scoring the candidate's match after the interview, means for comparatively evaluating multiple candidates based on the scoring results, means for efficiently securing personnel within the security department, and an emotion engine for evaluating the candidate's reliability and level of tension. This improves the efficiency of the hiring process in the security department and enables the prompt securing of suitable personnel.

[1221] An "enterprise" is a legal entity or organization established to carry out economic activities as an organization and provide goods and services.

[1222] "Recruitment requirements" refer to the standards, such as skills, experience, and qualifications, that a company requires of new employees.

[1223] A "job posting" is a document that provides information such as the type of job and conditions that a company is recruiting for to job seekers.

[1224] A "candidate" is a job seeker who applies for a job at a company and seeks employment.

[1225] "Document" refers to information in written form, such as a resume or curriculum vitae submitted by a candidate.

[1226] "Skills" refer to the knowledge and abilities required to efficiently carry out specific tasks or operations.

[1227] "Experience information" refers to specific information about the job or project a candidate has previously undertaken.

[1228] "Match rate" is an indicator that shows how closely a candidate's skills and experience match the company's hiring requirements.

[1229] "Footage" refers to video data that captures the candidate's appearance and behavior during an interview.

[1230] "Audio" refers to audio data recorded from the candidate's voice during the interview.

[1231] "Emotion analysis results" refer to the evaluation results of the candidate's emotional state obtained by analyzing video and audio data.

[1232] "Scoring" is the process of quantifying and evaluating a candidate's match.

[1233] "Comparative evaluation" is the process of evaluating multiple candidates based on their scoring results and selecting the most suitable candidate.

[1234] "Human resource acquisition" refers to a series of activities that a company undertakes to properly recruit and secure the human resources it needs.

[1235] "Credibility" is an indicator of whether a candidate's words, actions, and behavior are trustworthy.

[1236] "Tension level" is an indicator of the level of stress and anxiety a candidate feels during an interview.

[1237] An "emotion engine" is software or algorithms that analyze video and audio data to assess a candidate's emotional state.

[1238] This invention provides a "security interview assistant" system that enables companies to efficiently and accurately select the personnel they are looking for. The system for realizing this application example will be described in detail below.

[1239] First, the user (company recruiter) logs in to the recruitment portal site and enters the company's recruitment requirements. The entered recruitment requirements are then sent to the server via the terminal. The generative AI model on the server automatically generates a job posting based on these recruitment requirements.

[1240] Candidates upload documents such as their resumes and work histories to a recruitment portal site. These documents are then sent to a server via their device, where the server's natural language processing engine analyzes the documents to extract information about skills and experience. Based on this information, the candidate's match is evaluated.

[1241] When conducting a real-time interview, the user (interviewer) talks with the candidate via a video chat system. The device sends video and audio data to the server in real time. The server's video and audio analysis engines analyze the candidate's facial expressions and tone of voice, and use an emotion engine to assess their level of nervousness and confidence. This allows the interviewer to be presented with suitable question candidates and areas of concern.

[1242] Once the interview is over, the server combines the collected video and audio data, as well as the interviewer's evaluation, and uses an AI model to score each candidate's match. The scoring results are then adjusted, taking into account emotional data, to calculate a final score. Based on the scoring results, the server compares and evaluates multiple candidates, generates a report to select the most suitable candidate, and sends it to the device.

[1243] Specific applications include:

[1244] 1. Automatic job posting generation:

[1245] The server generates the most suitable job posting based on the security-related recruitment requirements entered by users who log in to the recruitment portal site. Example: Automatically generate a job posting for "candidates with specialized knowledge and work experience in network security with at least three years."

[1246] 2. Candidate characteristics analysis and match assessment:

[1247] The server's natural language processing engine analyzes the resumes and CVs uploaded by candidates, extracts security-related skills and experience, and determines the degree of match. For example, keywords such as "CISSP certification" and "security audit experience" are extracted.

[1248] 3. Real-time sentiment analysis during interviews:

[1249] The server analyzes the candidate's video and audio data collected through the video chat system to assess their emotional state in real time. It measures the candidate's level of nervousness and confidence and presents specific questions to the interviewer. For example, it presents questions such as, "Tell us about a security incident you were actually involved in in the past."

[1250] 4. Post-interview scoring and comparative evaluation:

[1251] After the interview, the server integrates the data collected and uses an AI model to quantify and evaluate the candidate's match. A final score is calculated that takes into account emotional data, and multiple candidates are compared and evaluated. For example, Candidate A is scored 90 points, and Candidate B is scored 85 points.

[1252] Real-time analysis using an emotion engine and advanced analysis using generative AI models is expected to streamline the recruitment of talent for security departments and significantly improve corporate recruitment processes.

[1253] Example prompt sentence:

[1254] Analyze and show the candidate's emotional state in real time: Analyze the candidate's nervousness and confidence level. Evaluate the candidate's emotional state in real time and suggest questions that will appear to the interviewer.

[1255] If the candidate seems nervous, suggest questions: for example, about past project experience.

[1256] This allows for the quick and accurate selection of personnel suitable for the security department.

[1257] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1258] Step 1:

[1259] A user logs in to a recruitment portal site. The user uses a terminal to input the company's recruitment requirements. An example of input data is "Network security specialist with at least three years of experience." The input data is sent to the server by the terminal.

[1260] Step 2:

[1261] The server uses a generative AI model to automatically generate a job posting based on the received hiring requirements data. The job posting includes requirements such as "CISSP certification" and "practical experience in network security." The generated job posting is sent from the server to the terminal, where it can be viewed by the user.

[1262] Step 3:

[1263] Candidates upload their resumes and work histories to a recruitment portal site. The terminal sends these document data to a server. An example of input data is "I have experience in security development in the financial industry."

[1264] Step 4:

[1265] The server inputs the received document data into a natural language processing (NLP) engine. The NLP engine analyzes the document and extracts skill and experience information. For example, keywords such as "CISSP certification" and "network security experience" are extracted. The extracted information is stored on the server.

[1266] Step 5:

[1267] The server uses a generative AI model to evaluate the candidate's match based on the extracted skills and experience information. The evaluation results are sent from the server to the device in the form of "high match" or "medium match," and can be viewed by the user.

[1268] Step 6:

[1269] At the start of the interview, the user (interviewer) talks with the candidate through a video call system. The terminal transmits video and audio data to the server in real time. For example, video frames and audio streams are input.

[1270] Step 7:

[1271] The server uses a video analysis engine and an audio analysis engine to analyze the video and audio data in real time. The candidate's facial expressions and tone of voice are extracted from the analysis results. The emotion engine uses this to evaluate the candidate's level of nervousness and confidence. The evaluation results are sent from the server to the terminal in real time, where the interviewer can check them.

[1272] Step 8:

[1273] During the interview, the server generates candidate questions and concerns based on the results of sentiment analysis in real time and displays them on the device. For example, if the candidate is feeling nervous, the server will suggest a question such as, "Tell us about your specific project experience."

[1274] Step 9:

[1275] After the interview, the server combines the collected video and audio data and the interviewer's evaluation. This data is input into an AI model, which scores the candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[1276] Step 10:

[1277] The server adjusts the scoring results based on the emotion data obtained by the emotion engine. After calculating the final score, a comparative evaluation is performed based on multiple candidates to select the most suitable candidate. The selection results are sent from the server to the terminal in report format, allowing the user (company recruiter) to make the final decision.

[1278] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1279] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1280] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1281] [Third embodiment]

[1282] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1283] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1284] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1285] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1286] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1287] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1288] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1289] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1290] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1291] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1292] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1293] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1294] System Overview

[1295] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatically creating job postings, analyzing candidates' characteristics and determining their match, analyzing real-time emotions during interviews, and scoring and comparative evaluation after interviews.

[1296] Program processing flow and specific examples

[1297] Automatic creation of job postings

[1298] 1. A user (company representative) logs in to a recruitment portal site and enters the recruitment requirements. For example, suppose the company is looking for a "software development engineer with at least three years of experience."

[1299] 2. The terminal sends the entered employment requirements to the server.

[1300] 3. The server inputs the received hiring requirements into the AI ​​model, which then generates the optimal job posting for the target and position based on past job postings and hiring history.

[1301] 4. The server sends the generated job posting to the terminal so that the user can check it. After checking, the user can make any necessary corrections and officially publish it.

[1302] Candidate characteristics analysis and match assessment

[1303] 1. The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[1304] 2. The terminal sends the uploaded document to the server.

[1305] 3. The server inputs the received document into a natural language processing (NLP) engine. The NLP engine analyzes the document and extracts skill and experience information. Examples of extracted skills include "financial industry" and "software development experience."

[1306] 4. The server uses the extracted information to evaluate the candidate's match using an AI model. The evaluation results are provided to the user (company representative), who determines whether the candidate has a high match.

[1307] Real-time analysis during the interview

[1308] 1. The user (interviewer) starts the interview and uses the video call system.

[1309] 2. The device transmits the interview video and audio data to the server in real time.

[1310] 3. The server uses a video analysis engine to analyze the candidate's emotions based on their facial expressions and tone of voice, for example, to determine their level of nervousness or confidence.

[1311] 4. The server displays the sentiment analysis results in real time on the interviewer's device and suggests suitable questions and areas of concern. For example, if the candidate is nervous, the server will prompt them to "Tell me more about your specific project experience."

[1312] Post-interview scoring and comparison

[1313] 1. After the interview is completed, the server integrates the collected video and audio data and the interviewer's evaluation.

[1314] 2. The server uses an AI model to score each candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[1315] 3. The server compares candidates based on the scoring results and determines the best candidate.

[1316] 4. The server generates a report containing the comparison results and sends it to the terminal. The user (company representative) makes the final hiring decision based on this report.

[1317] Technical effects

[1318] This system allows companies to streamline their recruitment processes and significantly reduce the burden on managers. In addition, AI-based data analysis enables objective and accurate candidate evaluation, enabling the rapid recruitment of suitable personnel. This is expected to improve a company's competitiveness and optimize the allocation of personnel.

[1319] The processing flow will be explained below.

[1320] Automatic creation of job postings

[1321] Step 1:

[1322] The user (company representative) logs in to the recruitment portal site and enters the recruitment requirements, for example, "Recruiting a software development engineer with at least three years of experience."

[1323] Step 2:

[1324] The terminal transmits the entered employment requirements to the server.

[1325] Step 3:

[1326] The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[1327] Step 4:

[1328] The AI ​​model generates optimal job descriptions, including requirements such as "Java or Python programming experience" and "team development experience."

[1329] Step 5:

[1330] The server sends the generated job posting to the terminal so that the user can check it.

[1331] Step 6:

[1332] After checking the job posting, the user makes any necessary corrections, makes a final check, and then publishes the job posting.

[1333] ---

[1334] Candidate characteristics analysis and match assessment

[1335] Step 1:

[1336] The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[1337] Step 2:

[1338] The terminal transmits the uploaded document to the server.

[1339] Step 3:

[1340] The server inputs the received document into a natural language processing (NLP) engine.

[1341] Step 4:

[1342] The NLP engine analyzes documents and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[1343] Step 5:

[1344] Based on the extracted information, the server uses an AI model to evaluate the candidate's match, for example, rating them as a "high match."

[1345] Step 6:

[1346] The server sends the evaluation results to the terminal so that the user (company representative) can check them.

[1347] ---

[1348] Real-time analysis during the interview

[1349] Step 1:

[1350] The user (interviewer) starts the interview and uses the video call system.

[1351] Step 2:

[1352] The terminal transmits the interview video and audio data to the server in real time.

[1353] Step 3:

[1354] The server uses a video analysis engine and an audio analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[1355] Step 4:

[1356] Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Please tell us more about your specific project experience."

[1357] Step 5:

[1358] The user (interviewer) conducts the interview based on the presented questions and asks additional questions if necessary.

[1359] ---

[1360] Post-interview scoring and comparison

[1361] Step 1:

[1362] The server integrates the collected video and audio data and the interviewer's evaluation after the interview is completed.

[1363] Step 2:

[1364] The server uses an AI model to score each candidate's match, for example, Candidate A might get 90 points, and Candidate B might get 85 points.

[1365] Step 3:

[1366] The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[1367] Step 4:

[1368] The server generates a report containing the comparison results and sends it to the terminal.

[1369] Step 5:

[1370] The user (company representative) makes the final hiring decision based on the received report.

[1371] ---

[1372] Through the above processing steps, the recruitment assistant AI system realizes an efficient and objective recruitment process, significantly reducing the burden on corporate managers.

[1373] Example 1

[1374] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1375] Traditional corporate recruitment processes require a wide range of manual tasks, such as creating job postings, evaluating candidates, conducting interviews, and scoring the results, which is time-consuming and labor-intensive. It is also difficult to analyze candidates' characteristics and emotions in real time, making objective evaluations difficult. Therefore, there is a need to reduce the burden on recruiters and achieve a more efficient and objective recruitment process.

[1376] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1377] In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents and extracting skills and experience information using a natural language processing engine, means for evaluating the candidate's match based on the skills and experience information, means for analyzing video and audio during the interview in real time and displaying the results of a sentiment analysis, means for presenting suitable question candidates and concerns during the interview, means for scoring the candidate's match after the interview, and means for comparatively evaluating multiple candidates based on the scoring results, thereby enabling efficiency improvement and objective evaluation of the entire hiring process.

[1378] "Employment requirements" refer to the specific conditions, such as skills, experience, educational background, and qualifications, that a company seeks in its employees.

[1379] A "job posting" is a document containing recruitment information published by a company, which lists the job types, conditions, and work content.

[1380] "Documents" refers to documents submitted by candidates, such as resumes and application forms.

[1381] A "natural language processing engine" is software or a system that analyzes the content of documents and extracts skill and experience information.

[1382] "Skills and experience information" is information that indicates the candidate's abilities and past work experience.

[1383] "Match" is an evaluation that indicates how closely a candidate's characteristics match the company's hiring requirements.

[1384] "Real-time analysis of video and audio" refers to a technology that analyzes video and audio collected during an interview on the spot.

[1385] "Emotion analysis results" are data showing the emotional state of a candidate obtained by analyzing their facial expressions and tone of voice.

[1386] "Suggested Questions and Concerns" present questions that interviewers should ask candidates and points to be aware of during the interview.

[1387] "Scoring" refers to the process of assigning a score to the interview results and the candidate's characteristics to evaluate them.

[1388] "Comparative evaluation" means comparing multiple candidates based on scoring results.

[1389] System Overview

[1390] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatically creating job postings, analyzing candidates' characteristics and determining their match, analyzing real-time emotions during interviews, and scoring and comparative evaluation after interviews.

[1391] Automatic creation of job postings

[1392] The user (company representative) accesses the login page using their company's account information and enters specific hiring requirements, such as "software development engineer with at least three years of experience." The device collects the information entered by the user in real time and sends it securely to the server using HTTPS. The server converts the received hiring requirements into an appropriate format and inputs it into an AI model using TensorFlow. The model references a past database and generates an optimal job posting. The server sends the generated job posting in JSON format to the device, which displays it on the user's screen. The user checks the content on the screen, modifies the job posting if necessary, and clicks the "Publish" button, which officially posts the job information on the portal site.

[1393] Examples:

[1394] If a user enters a job requirement for "an engineer with 5+ years of experience in front-end development," the server generates a corresponding job posting and provides it as "React.js developer with 5+ years of experience wanted." The user can then review the generated job posting, modify it, and publish it.

[1395] Example prompt:

[1396] "Please create a job posting for an engineer with at least 5 years of experience in front-end development."

[1397] Candidate characteristics analysis and match assessment

[1398] Users (candidates) upload their resumes and application forms to the system in an appropriate format (PDF or Word). The terminal scans the uploaded files and sends them to the server. The server launches a natural language processing (NLP) engine using PyTorch to analyze the text within the document. For example, it extracts keywords such as "financial industry" and "software development experience." The server inputs the extracted information into a pre-prepared evaluation algorithm and numerically evaluates the match between the candidate's skills and the company's requirements. The evaluation results are sent to the terminal, and the user can view a detailed evaluation of the candidate on the screen.

[1399] Examples:

[1400] If a candidate states that they have "software development experience in the financial industry" in the past, the server will extract "financial industry" and "software development experience" and compare them with the company's job requirements to evaluate the match as 90%.

[1401] Example prompt:

[1402] "Analyze the resumes of candidates with software development experience in the financial industry to determine the degree of match."

[1403] Real-time sentiment analysis during interviews

[1404] The user (interviewer) initiates a video call with a candidate via Zoom or Microsoft Teams. The device captures the video and audio data of the video call in real time and sends it to the server. The server runs a video analysis engine such as OpenCV to analyze the candidate's facial expressions and tone of voice. For example, it analyzes their level of nervousness and confidence. The server then creates candidate questions corresponding to the analysis results and displays them in real time on the interviewer's device. The interviewer can use this information to ask the candidate appropriate questions.

[1405] Examples:

[1406] If sentiment analysis reveals that a candidate is nervous during an interview, the server will display "High nervousness" and suggest questions to the interviewer, such as "Tell me more about your specific project experience."

[1407] Example prompt:

[1408] "Perform real-time sentiment analysis of candidate tension during interviews and suggest appropriate questions."

[1409] Post-interview scoring and comparison

[1410] The server combines the video and audio data collected during the interviews, as well as the interviewers' manual evaluations, and stores them as a single dataset. The server inputs the combined dataset into an AI model, which evaluates each candidate's skills, aptitude, and expressions during the interview to create a score. For example, candidate A may receive a score of 90, while candidate B receives a score of 85. The server compares the scoring results and determines the most suitable candidate, taking into account algorithms and past hiring performance data. The server generates a detailed report including the final evaluation and comparison results and sends it to the device. The user makes the final hiring decision based on this report. The user reviews the report displayed on the device and selects the most suitable candidate. Once the final decision is made, the server sends an offer of employment.

[1411] Technical effects

[1412] This system allows companies to streamline their recruitment processes and significantly reduce the burden on managers. In addition, AI-based data analysis enables objective and accurate candidate evaluation, enabling the rapid recruitment of suitable personnel. This is expected to improve a company's competitiveness and optimize the allocation of personnel.

[1413] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1414] Step 1:

[1415] A user logs in to a recruitment portal site and enters their recruitment requirements.

[1416] Input: User inputs recruitment requirements (e.g., "Software development engineer with at least 3 years of experience")

[1417] What happens: The user logs in using their company's account information and enters their job requirements in the text fields.

[1418] Step 2:

[1419] The terminal transmits the entered employment requirements to the server.

[1420] Input: Recruitment requirements data

[1421] Data processing / calculation: The device collects input information in real time and sends it securely to the server using HTTPS

[1422] Output: Recruitment requirements data sent to the server

[1423] Specific operation: The terminal registers data in the transmission queue and sends it to the server via HTTPS communication.

[1424] Step 3:

[1425] The server inputs the recruitment requirements received into the AI ​​model and generates a job posting.

[1426] Input: Recruitment requirements data received by the server

[1427] Data processing / calculation: The server converts the data into the appropriate format, inputs it into an AI model using TensorFlow, and generates a job posting.

[1428] Output: Generated job posting data

[1429] Specific operation: The server retrieves relevant information from the database and inputs it into the AI ​​model to generate a job posting.

[1430] Step 4:

[1431] The server sends the generated job posting to the terminal and the user confirms it.

[1432] Input: Generated job posting data

[1433] Data processing / calculation: The server converts the generated job posting into JSON format and sends it to the terminal.

[1434] Output: The job posting displayed to the user

[1435] Specific operation: The server sends JSON format data to the terminal, which parses it and displays it on the user's screen.

[1436] Step 5:

[1437] The user modifies and publishes the job posting.

[1438] Input: User-modified job posting data

[1439] Data processing / calculation: The user edits the job posting details on the screen and clicks the "Publish" button.

[1440] Output: Published job postings

[1441] Specific operation: The user checks the content, makes corrections, and then presses the publish button, and the job information is posted on the portal site.

[1442] Step 6:

[1443] The user (candidate) uploads their resume and application form.

[1444] Input: Candidates upload their resumes and application forms

[1445] Data processing / calculation: Upload document data to the system

[1446] Output: Uploaded document data

[1447] What happens: Candidates follow the system instructions and upload documents using the file selection button.

[1448] Step 7:

[1449] The terminal sends the uploaded document to the server.

[1450] Input: Uploaded document data

[1451] Data processing / calculation: The device scans the file and sends it to the server

[1452] Output: Document data sent to the server

[1453] Specific operation: The device temporarily stores the uploaded file and securely sends it to the server.

[1454] Step 8:

[1455] The server inputs the document into an NLP engine for analysis.

[1456] Input: Document data sent to the server

[1457] Data processing / calculation: The server runs a natural language processing (NLP) engine using PyTorch to perform analysis. Specifically, it extracts keywords such as "financial industry" and "software development experience."

[1458] Output: Parsed skills and experience information

[1459] How it works: The server inputs documents into the NLP engine and extracts important skills and experience information.

[1460] Step 9:

[1461] The server uses the analysis results to evaluate the degree of match using an AI model.

[1462] Input: Parsed skills and experience information

[1463] Data processing / calculation: The server inputs the extracted information into an evaluation algorithm and evaluates the candidate's match numerically.

[1464] Output: Evaluation results

[1465] Specific operation: The server scores the extracted information based on an evaluation algorithm.

[1466] Step 10:

[1467] The server provides the evaluation results to the user.

[1468] Input: Evaluation result data

[1469] Data processing / calculation: Evaluation results sent to the terminal

[1470] Output: Evaluation results displayed on the user's screen

[1471] Specific operation: The server sends the evaluation results in JSON format to the terminal, and the user confirms them.

[1472] Step 11:

[1473] The user (interviewer) starts the interview using the video call system.

[1474] Input: Start the video call system

[1475] Data processing / calculation: Capture of interview video and audio data

[1476] Output: Captured video and audio data

[1477] What happens: The interviewer will start a video call using Zoom or Microsoft Teams.

[1478] Step 12:

[1479] The terminal transmits the interview video and audio data to the server.

[1480] Input: Captured video and audio data

[1481] Data processing / calculation: Send to server in real time

[1482] Output: Video and audio data sent to the server

[1483] Specific operation: The device captures video and audio data in real time and sends it to the server.

[1484] Step 13:

[1485] The server performs emotion analysis using a video analysis engine.

[1486] Input: Video and audio data sent to the server

[1487] Data processing / calculation: Emotion analysis is performed using OpenCV etc. to analyze tension and confidence.

[1488] Output: Sentiment analysis results

[1489] Specific operation: The server inputs video and audio data into an analysis engine to evaluate the emotional state.

[1490] Step 14:

[1491] The server displays the analysis results and suitable question candidates on the device.

[1492] Input: Sentiment analysis results

[1493] Data processing / calculation: Generate suitable question candidates and send them to the device

[1494] Output: Question candidates and sentiment analysis results displayed on the user's screen

[1495] Specific operation: The server generates candidate questions based on the analysis results and sends them to the device, which then displays them to the user.

[1496] Step 15:

[1497] The server consolidates the collected data.

[1498] Input: Video and audio data collected during the interview, and manual assessment by the interviewer

[1499] Data processing / calculation: Data integration and storage

[1500] Output: A consolidated dataset

[1501] Specific operation: The server consolidates and centrally manages all data.

[1502] Step 16:

[1503] The server scores each candidate using an AI model.

[1504] Input: Integrated dataset

[1505] Data processing / calculation: Input data into the AI ​​model to generate a score for each candidate (e.g., candidate A scores 90, candidate B scores 85)

[1506] Output: Scoring results

[1507] How it works: The server inputs data into the AI ​​model and generates a score.

[1508] Step 17:

[1509] The server compares the candidates to determine the best candidate.

[1510] Input: Scoring results

[1511] Data processing / calculation: Use comparison algorithms to determine the best candidates

[1512] Output: The best candidate is identified

[1513] Specific operation: The server compares the scoring results and selects the most suitable candidate.

[1514] Step 18:

[1515] The server sends the comparison results as a report to the terminal.

[1516] Input: The best candidate's results

[1517] Data processing / calculation: Generate detailed reports and send them to your device

[1518] Output: The report that is displayed on the user's screen

[1519] Specific operation: The server generates a report based on the evaluation results and sends it to the terminal, which then displays it to the user.

[1520] Step 19:

[1521] The user makes the final hiring decision based on the report.

[1522] Input: Best candidate report

[1523] Data processing / calculation: Making decisions based on reports

[1524] Output: Final hiring decision

[1525] Specific actions: The user reviews the report, selects the best candidates, and sends out job offers.

[1526] (Application example 1)

[1527] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1528] In the recruitment process, analyzing candidate characteristics and emotions during interviews is important, but there is no system that utilizes this data in real time to evaluate risk factors. As a result, companies are forced to rely on subjective judgment when selecting appropriate candidates, making efficient recruitment difficult. In addition, it is difficult to detect security risks early in post-recruitment activities.

[1529] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1530] In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and displaying the emotion analysis results, means for analyzing the candidate's behavior in real time during the interview and performing a risk assessment, means for scoring the candidate's match after the interview, and means for comparatively assessing multiple candidates based on the scoring results. This enables a more objective and efficient hiring process based on real-time emotion and risk analysis.

[1531] "Means for inputting a company's recruitment requirements" refers to an interface that allows a company to input the conditions, qualifications, skills, etc. of the personnel it wishes to hire into the system.

[1532] The "means for automatically generating a job posting based on the hiring requirements" refers to an algorithm and system that creates optimal job advertisements and job postings based on the hiring requirements entered.

[1533] "Means for receiving documents submitted by candidates" refers to a function for receiving resumes and application forms uploaded by candidates to the system.

[1534] The "means for analyzing the document and extracting skill and experience information" is a function that uses natural language processing technology to extract information such as the candidate's skills and work history from the received document.

[1535] The "means for evaluating the degree of match of a candidate based on the skills and experience information" refers to an algorithm and system that compares the extracted skills and experience information and evaluates the degree of suitability of a candidate for the recruitment requirements.

[1536] "A means of analyzing video and audio during interviews in real time and displaying the results of emotional analysis" is a function that instantly processes video and audio collected during interviews, analyzes the candidate's facial expressions and tone of voice, and displays their emotional state.

[1537] "Means for analyzing candidate behavior in real time during interviews and conducting risk assessments" refers to a function that observes candidate actions and behaviors occurring during interviews in real time and assesses risk based on that information.

[1538] "Means for scoring candidate match after interview" refers to algorithms and systems that quantify and evaluate a candidate's suitability based on data collected after the interview.

[1539] The "means for comparatively evaluating a plurality of candidates based on the scoring results" is a function for comparing the evaluation results of scored candidates and selecting the most suitable candidate.

[1540] This invention provides a recruitment assistant AI system that integrates multiple functions to streamline a company's recruitment process and reduce the burden on managers. The configuration and operation of this system are described below.

[1541] Automatic creation of job postings

[1542] The server automatically generates job postings based on information received through the company's recruitment requirements input method. At this time, the server uses a generative AI model based on past recruitment data and recruitment performance to create the optimal job posting. The user can check this job posting via their device, make any necessary corrections, and publish the job information.

[1543] Candidate characteristics analysis and match assessment

[1544] The server receives the resumes and application forms submitted by candidates. Based on this, it analyzes the documents using natural language processing (NLP) technology to extract skill and experience information. The server then inputs the extracted information into a generative AI model to evaluate the candidate's match.

[1545] Real-time sentiment analysis and risk assessment during interviews

[1546] The user (interviewer) uses smart glasses during the interview to collect video and audio data. The server analyzes this data in real time, analyzing the candidate's facial expressions and tone of voice to understand their emotional state. It also analyzes the candidate's behavior in real time during the interview and performs a risk assessment. This data is displayed on the interviewer's device and serves as a reference for how to respond.

[1547] Post-interview scoring and comparative evaluation

[1548] After the interviews are completed, the server integrates the collected data and scores the candidates' match. It uses a generative AI model to evaluate each candidate's suitability and compare multiple candidates. It then generates a report of the best candidates and provides it to the recruiter.

[1549] Hardware and software used

[1550] This system mainly uses the following hardware and software:

[1551] Hardware: Smart glasses, devices (PC, tablet, etc.)

[1552] Software: Python, OpenCV, Keras, NLP engine, generative AI model

[1553] Examples of specific examples and prompts

[1554] For example, if a company is looking for a "software development engineer with at least three years of experience," the server will generate the most suitable job posting based on this information and analyze the candidate's submitted documents. During the interview, the smart glasses will analyze the candidate's behavior in real time and perform a risk assessment.

[1555] Example prompt sentence:

[1556] "Develop an application that uses camera footage to analyze candidates' facial expressions in real time, determine their emotions, and conduct a risk assessment. Display the results in real time."

[1557] In this way, the present invention assists in the entire recruitment process, allowing companies to efficiently recruit the right talent.

[1558] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1559] Step 1:

[1560] The user (company representative) enters the recruitment requirements.

[1561] Input: Job requirements (e.g., "Software development engineer with at least 3 years of experience")

[1562] Output: The entered recruitment requirements are sent to the server.

[1563] Specific operations: The company representative logs in to the recruitment portal site and enters the necessary recruitment requirements in detail into the form.

[1564] Step 2:

[1565] The server automatically generates a job posting based on the received hiring requirements.

[1566] Input: Recruitment requirements

[1567] Output: Generated job posting

[1568] Specific operation: The server uses the recruitment requirement data input into the generative AI model, refers to past cases, and creates the most appropriate job posting.

[1569] Step 3:

[1570] The server sends the generated job posting to the user's (company representative's) terminal.

[1571] Input: Generated job posting

[1572] Output: Job posting confirmation screen

[1573] Specific operation: The server sends the created job posting to the company's employee's terminal and displays a screen where the employee can check and edit the job posting.

[1574] Step 4:

[1575] The user uploads the candidate's documents (resume and application form).

[1576] Input: resume and application form

[1577] Output: Uploaded documents are sent to the server.

[1578] What happens: Candidates use a web portal to upload their resume and application form.

[1579] Step 5:

[1580] The server analyzes the uploaded documents using a natural language processing (NLP) engine.

[1581] Input: resume and application form

[1582] Output: Extraction results of skills and experience information

[1583] Specific operation: The server uses an NLP engine to analyze the contents of the document and extract skill and experience information such as "software development experience in the financial industry."

[1584] Step 6:

[1585] The server evaluates the match of the candidate based on the extracted information.

[1586] Input: Extracted skills and experience information

[1587] Output: Candidate match rating

[1588] How it works: The server uses a generative AI model to match the extracted information with the job requirements and score the match.

[1589] Step 7:

[1590] The user (interviewer) uses smart glasses during the interview to collect video and audio of the candidate in real time.

[1591] Input: Video and audio data from the interview

[1592] Output: Real-time data sent to the server

[1593] How it works: The interviewer puts on the smart glasses and begins the interview. The glasses' camera and microphone collect video and audio of the candidate in real time.

[1594] Step 8:

[1595] The server analyzes the data received in real time and displays an assessment of emotional state and behavioral risk.

[1596] Input: Video and audio data from the interview

[1597] Output: Real-time display of emotional state and risk assessment

[1598] Specific operation: The server inputs the received data into an analysis engine, analyzes the candidate's emotional state (e.g., level of tension, confidence), and displays the evaluation results on the interviewer's terminal.

[1599] Step 9:

[1600] The server consolidates the data collected after the interview and scores the candidate's match.

[1601] Input: Video and audio data collected during the interview, and interviewer evaluation

[1602] Output: Match score

[1603] How it works: The server consolidates the collected data and uses a generative AI model to score candidates for match.

[1604] Step 10:

[1605] The server performs comparative evaluation based on the scoring results of multiple candidates and generates a report.

[1606] Input: Match score for each candidate

[1607] Output: Comparative evaluation report

[1608] Specific operation: The server compares the scoring results of each candidate, generates a report to select the most suitable candidate, and provides it to the recruiter.

[1609] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1610] System Overview

[1611] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. In particular, by incorporating an emotion engine, it enables more precise analysis that takes into account the candidate's emotional state during and after the interview. This system supports each stage of the recruitment process, including automatic creation of job postings, analysis of candidate characteristics and match determination, real-time emotion analysis using the emotion engine, and post-interview scoring and comparative evaluation.

[1612] Program processing flow and specific examples

[1613] Automatic creation of job postings

[1614] 1. A user (company representative) logs in to a recruitment portal site and enters the recruitment requirements. For example, they might enter "Recruiting a software development engineer with at least three years of experience."

[1615] 2. The terminal sends the entered employment requirements to the server.

[1616] 3. The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[1617] 4. The AI ​​model generates the most suitable job description, including requirements such as "Java and Python programming experience" and "team development experience."

[1618] 5. The server sends the generated job posting to the terminal so that the user can view it.

[1619] 6. After checking the job posting, the user will make any necessary corrections, make a final confirmation, and then publish it.

[1620] Candidate characteristics analysis and match assessment

[1621] 1. The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience developing software in the financial industry."

[1622] 2. The terminal sends the uploaded document to the server.

[1623] 3. The server inputs the received document into a natural language processing (NLP) engine.

[1624] 4. The NLP engine analyzes the document and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[1625] 5. Based on the extracted information, the server uses an AI model to evaluate the candidate's match, e.g., a "high match."

[1626] 6. The server sends the evaluation results to the terminal so that the user (company representative) can check them.

[1627] Real-time analysis during the interview

[1628] 1. The user (interviewer) starts the interview and uses the video call system.

[1629] 2. The device transmits the interview video and audio data to the server in real time.

[1630] 3. The server uses a video analysis engine and a voice analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[1631] 4. The server uses an emotion engine to perform a more detailed analysis of the candidate's emotional state, assessing their level of nervousness, confidence, etc.

[1632] 5. Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Tell me more about your specific project experience."

[1633] 6. The user (interviewer) conducts the interview based on the questions presented and asks additional questions if necessary.

[1634] Post-interview scoring and comparison

[1635] 1. The server integrates the video and audio data collected after the interview and the interviewer's evaluation.

[1636] 2. The server uses an AI model to score each candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[1637] 3. The server adjusts the scoring results based on the emotion data obtained by the emotion engine, and calculates the final score taking into account the level of tension and confidence from the emotion data.

[1638] 4. The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[1639] 5. The server generates a report containing the comparison results and sends it to the terminal.

[1640] 6. The user (company representative) makes the final hiring decision based on the received report.

[1641] Technical effects

[1642] This system enables companies to streamline their recruitment processes and significantly reduce the burden on managers. In particular, by analyzing emotional data using an emotion engine, it is possible to grasp the true state of candidates during interviews, enabling more accurate evaluations. This will enable companies to quickly hire the right talent, which is expected to improve a company's competitiveness and optimize personnel allocation.

[1643] The processing flow will be explained below.

[1644] Automatic creation of job postings

[1645] Step 1:

[1646] The user (company representative) logs in to the recruitment portal site and enters the recruitment requirements, for example, "Recruiting a software development engineer with at least three years of experience."

[1647] Step 2:

[1648] The terminal transmits the entered employment requirements to the server.

[1649] Step 3:

[1650] The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[1651] Step 4:

[1652] The AI ​​model generates optimal job postings, such as those that include requirements such as "Java or Python programming experience" and "team development experience."

[1653] Step 5:

[1654] The server sends the generated job posting to the terminal, where it is displayed so that the user can check it.

[1655] Step 6:

[1656] The user checks the generated job posting, modifies it as necessary, and then publishes it.

[1657] ---

[1658] Candidate characteristics analysis and match assessment

[1659] Step 1:

[1660] Users (candidates) upload their resumes and application forms to a recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[1661] Step 2:

[1662] The terminal transmits the uploaded document to the server.

[1663] Step 3:

[1664] The server inputs the received document into a natural language processing (NLP) engine.

[1665] Step 4:

[1666] The NLP engine analyzes documents and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[1667] Step 5:

[1668] Based on the extracted information, the server uses an AI model to evaluate the candidate's match, for example, rating them as a "high match."

[1669] Step 6:

[1670] The server sends the evaluation results to the terminal and displays them so that the user (company representative) can check them.

[1671] ---

[1672] Real-time analysis during the interview

[1673] Step 1:

[1674] The user (interviewer) starts the interview and uses the video call system.

[1675] Step 2:

[1676] The terminal transmits the interview video and audio data to the server in real time.

[1677] Step 3:

[1678] The server uses a video analysis engine and an audio analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[1679] Step 4:

[1680] The server uses an emotion engine to further analyze the candidate's emotional state, assessing, for example, their level of nervousness and confidence.

[1681] Step 5:

[1682] Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Please tell us more about your specific project experience."

[1683] Step 6:

[1684] The user (interviewer) conducts the interview based on the presented questions and asks additional questions as necessary.

[1685] ---

[1686] Post-interview scoring and comparison

[1687] Step 1:

[1688] The server integrates the collected video and audio data and the interviewer's evaluation after the interview is completed.

[1689] Step 2:

[1690] The server uses an AI model to score each candidate's match, for example, Candidate A might get 90 points, and Candidate B might get 85 points.

[1691] Step 3:

[1692] The server adjusts the scoring results based on the emotion data obtained by the emotion engine, and calculates a final score that takes into account the level of tension and confidence from the emotion data.

[1693] Step 4:

[1694] The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[1695] Step 5:

[1696] The server generates a report containing the comparison results and sends it to the terminal.

[1697] Step 6:

[1698] The user (company representative) makes the final hiring decision based on the received report.

[1699] ---

[1700] Through the above processing steps, this system streamlines the entire recruitment process, reducing the burden on managers, and improves the accuracy of candidate aptitude assessment through detailed emotion analysis using an emotion engine.

[1701] Example 2

[1702] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1703] Traditional recruitment processes are time-consuming and labor-intensive, and rely on human evaluation, making them difficult to operate efficiently, especially when there are a large number of applicants. It is also difficult to determine a candidate's actual emotional state during the interview, making evaluations subjective. This increases the risk of delays in hiring the right talent and increases the likelihood of mismatches. Therefore, there is a need for a way to streamline the entire recruitment process and improve accuracy.

[1704] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1705] In this invention, the server includes means for inputting hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and providing emotion analysis results, means for scoring the candidate's match after the interview, means for comparatively evaluating multiple candidates based on the scoring results, means for adjusting the scoring results using emotion data, and means for selecting the most suitable candidate based on the adjusted scoring results. This makes the entire hiring process more efficient and enables more accurate and objective evaluations.

[1706] The "means for inputting recruitment requirements" is a method for providing an interface for users to input the conditions and needs of a company regarding recruitment.

[1707] "Means for automatically generating a job posting based on the hiring requirements" refers to a method for automatically creating an appropriate job posting using an AI model based on the hiring requirements entered.

[1708] The "means for receiving documents submitted by candidates" refers to a method for receiving documents such as resumes and application forms uploaded by candidates onto a server.

[1709] The "means for analyzing the document and extracting skill and experience information" is a method for analyzing the received document using natural language processing technology and extracting information such as the candidate's skills and work experience.

[1710] "Means for evaluating the degree of match of a candidate based on the skills and experience information" refers to a method for evaluating the degree to which a candidate matches the company's recruitment requirements based on the extracted skills and experience information.

[1711] "Means for analyzing video and audio data during an interview in real time and providing emotional analysis results" refers to a method for analyzing video and audio data acquired during an interview in real time, determining the emotional state of the candidate, and providing the results to the interviewer.

[1712] "Method of scoring candidate match after interview" is a method of scoring each candidate's suitability based on interview data.

[1713] The "means for comparatively evaluating a plurality of candidates based on the scoring results" refers to a method for comparing a plurality of candidates based on the scoring results and selecting the candidate with the highest degree of suitability.

[1714] "Means for adjusting scoring results using emotional data" refers to a method for correcting scoring results based on emotional data obtained during the interview and evaluating the final degree of fit.

[1715] The "means for selecting the most suitable candidate based on the adjusted scoring results" is a method for selecting the most suitable candidate based on the adjusted scoring results.

[1716] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatic creation of job postings, candidate characteristic analysis and match assessment, real-time emotion analysis using an emotion engine, and post-interview scoring and comparative evaluation. In particular, the incorporation of an emotion engine enables more precise analysis that takes into account the candidate's emotional state during and after the interview.

[1717] Hardware and software used

[1718] The server will be a high-performance cloud server. Specifically, cloud platforms such as Amazon Web Services (AWS) and Microsoft Azure are considered. Standard PCs, tablets, and smartphones can be used as terminals. The following software will be used:

[1719] Natural language processing model (NLP engine): SpaCy or Stanford NLP

[1720] Generative AI models: GPT-3 and BERT

[1721] Video analysis engine: OpenCV

[1722] Voice analysis engine: Librosa

[1723] Sentiment analysis engine: AI model equipped with FaceAPI and Sentiment Analysis

[1724] Video calling system: Zoom or Microsoft Teams

[1725] Program processing

[1726] Automatic creation of job postings

[1727] The user (company representative) logs in to the recruitment portal site and enters their hiring requirements. For example, they may enter requirements such as "We are looking for a software development engineer with at least three years of experience." The device then sends the entered data to the server. The server then inputs the received data into a generative AI model, compares it with past job posting data, and generates the most suitable job posting. The generated job posting is then sent to the device, where it can be viewed by the user. Any necessary corrections are made, and the job posting is published after a final confirmation.

[1728] Candidate characteristics analysis and match assessment

[1729] The user (candidate) uploads their resume and application form to a recruitment portal site. For example, they may upload a document stating that they have previous experience developing software in the financial industry. The device then sends the uploaded document to the server. The server inputs the received document into an NLP engine, analyzes the document, and extracts skill and experience information. Based on the extracted information, an AI model is used to evaluate the candidate's match. The evaluation results are sent to the device and can be viewed by the user.

[1730] Real-time analysis during the interview

[1731] The user (interviewer) begins the interview using a video call system. The device sends the interview video and audio data to the server in real time. The server uses a video analysis engine and an audio analysis engine to analyze the video and audio data in real time. The emotion analysis engine is used to perform a detailed analysis of the candidate's emotional state and evaluate their level of nervousness and confidence. The analysis results are displayed on the interviewer's device as suitable question suggestions and areas of concern. For example, if the candidate is nervous, a question such as "Tell me more about your specific project experience" will be presented.

[1732] Post-interview scoring and comparative evaluation

[1733] Once the interview is over, the server combines the collected video and audio data and the interviewer's evaluation. It then uses an AI model to score each candidate's match. For example, candidate A might be rated 90 points, and candidate B 85 points. The scoring results are then adjusted based on emotional data to calculate the final score. Multiple candidates are compared and evaluated to determine the most suitable candidate. A report containing the final comparison results is generated and sent to the device. The user (company representative) makes the final hiring decision based on this report.

[1734] Examples of concrete examples and prompts

[1735] For example, use the following prompt:

[1736] 1. "We are looking for a software development engineer with at least 3 years of experience. Experience with Java or Python programming and team development is essential."

[1737] 2. "I have previous software development experience in the financial industry."

[1738] 3. "Tell me more about your specific project experience."

[1739] This will enable companies to streamline the recruitment process and significantly reduce the burden on managers. In particular, analyzing emotional data using an emotion engine will enable candidates to understand their true state during interviews, enabling more accurate evaluations.

[1740] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1741] Step 1:

[1742] A user logs in to a recruiting portal site.

[1743] Specific operation: The user accesses the login screen of the portal site using their company account and enters their authentication information. The server verifies the entered authentication information and authenticates the user. If authentication is successful, the user can access the main screen of the recruitment portal site.

[1744] Step 2:

[1745] The user enters the hiring requirements.

[1746] Input: Requirements such as job type, required skills, years of experience, etc.

[1747] Specific operation: The user enters specific employment requirements into a form on the site, such as "Seeking a software development engineer with at least three years of experience. Experience in Java or Python programming and team development is essential." The terminal then displays the entered information and allows the user to confirm it.

[1748] Step 3:

[1749] The terminal transmits the input data to the server.

[1750] Input: User-entered recruitment requirements

[1751] Output: Requirements data sent to the server

[1752] Specific operation: The terminal sends the recruitment requirement data to the server using an HTTP POST request. The server parses the received data and stores it in an appropriate format.

[1753] Step 4:

[1754] The server inputs the data into a generative AI model for analysis.

[1755] Input: Recruitment requirements data

[1756] Output: Generated job posting

[1757] How it works: The server inputs the received recruitment requirements data into a natural language processing model such as GPT-3 or BERT. The AI ​​model then references past recruitment data and generates a job posting that best fits the input requirements.

[1758] Step 5:

[1759] The server sends the generated job posting to the terminal.

[1760] Input: Generated job posting

[1761] Output: Job posting displayed on terminal

[1762] Specific operation: The server sends the generated job posting data to the terminal so that the user can check it. The terminal displays the received data and provides an interface that allows the user to check and modify it.

[1763] Step 6:

[1764] The user reviews and publishes the job posting.

[1765] Specific operation: The user checks the job posting displayed on the device and makes any necessary corrections. After completing the final check, the user presses the "Publish" button to publish the job posting. The device then sends the published information to the server and publishes the job posting.

[1766] Step 7:

[1767] The user uploads a resume or application form.

[1768] Input: resume and application form

[1769] Specific operation: The user (candidate) uses the upload screen on the recruitment portal site to select and upload the necessary documents such as resumes and application forms. The terminal then sends the selected documents to the server.

[1770] Step 8:

[1771] The terminal sends the document to the server.

[1772] Input: Uploaded document

[1773] Output: Document data stored on the server

[1774] Specific operation: The terminal sends the document data uploaded by the user to the server, and the server stores the received data in an appropriate format.

[1775] Step 9:

[1776] The server inputs the document into the NLP engine.

[1777] Input: Uploaded document data

[1778] Output: Skills and experience information

[1779] How it works: The server inputs the received document data into a natural language processing engine such as SpaCy or Stanford NLP for analysis. The NLP engine extracts keywords related to skills and work experience from the document.

[1780] Step 10:

[1781] The server evaluates the match of the candidate based on the extracted information.

[1782] Input: Skills and experience information

[1783] Output: Candidate match evaluation results

[1784] Specific operation: The server inputs the skills and experience information extracted by the NLP engine into the generative AI model to evaluate the candidate's match with the recruitment requirements. The evaluation result (e.g., "high match") is scored based on the company's recruitment criteria.

[1785] Step 11:

[1786] The server transmits the evaluation results to the terminal.

[1787] Input: Match evaluation result

[1788] Output: Evaluation results displayed on the terminal

[1789] Specific operation: The server sends the evaluation results to the terminal so that the user can check them. The terminal displays the received evaluation results and notifies the user.

[1790] Step 12:

[1791] The user starts the interview.

[1792] Specific operation: The user (interviewer) starts an interview with a candidate using a video call system such as Zoom or Microsoft Teams. The user accesses the video call system from their device and invites the candidate to participate.

[1793] Step 13:

[1794] The device transmits the interview video and audio data to the server in real time.

[1795] Input: Video and audio data from the interview

[1796] Output: Real-time data sent to the server

[1797] Specific operation: The device streams video and audio data captured during the interview to the server in real time, and the server prepares the infrastructure to process the received data appropriately.

[1798] Step 14:

[1799] The server analyzes the data and uses an emotion engine to analyze the emotional state.

[1800] Input: Real-time video and audio data

[1801] Output: Analysis of the candidate's emotional state

[1802] Specific operation: The server analyzes video and audio data using a video analysis engine (OpenCV) and an audio analysis engine (Librosa). The server inputs the analysis results into an emotion engine, which analyzes the candidate's facial expressions and tone of voice in real time.

[1803] Step 15:

[1804] The server sends potential questions and concerns to the device.

[1805] Input: Sentiment analysis results

[1806] Output: Possible questions and concerns displayed on the device

[1807] Specific operation: The server generates suitable candidate questions and concerns based on the results of sentiment analysis. The server then sends the generated information to the terminal in real time and presents it to the user (interviewer).

[1808] Step 16:

[1809] The user conducts the interview based on the questions presented.

[1810] Specific operation: The user conducts the interview while referring to the candidate questions and concerns displayed on the device. If necessary, the user asks additional questions and collects the candidate's answers.

[1811] Step 17:

[1812] The server consolidates the data after the interview is completed.

[1813] Input: Interview video, audio data, interviewer notes

[1814] Output: Consolidated interview data

[1815] Specific operation: After the interview is completed, the server integrates the collected video and audio data, as well as the interviewer's evaluation notes. The server then integrates and centrally manages this data.

[1816] Step 18:

[1817] The server uses an AI model to score each candidate's match.

[1818] Input: Consolidated interview data

[1819] Output: Match score for each candidate

[1820] Specific operation: The server inputs the integrated interview data into the generative AI model and scores each candidate's match. For example, candidate A might receive a score of 90, and candidate B might receive a score of 85.

[1821] Step 19:

[1822] The server adjusts the scoring results.

[1823] Input: Emotion data, scoring results

[1824] Output: Adjusted score

[1825] How it works: The server modifies the scoring results based on emotional data obtained during the interview, for example, adjusting the final score to take into account the level of nervousness or confidence.

[1826] Step 20:

[1827] The server compares and evaluates multiple candidates.

[1828] Input: Adjusted score

[1829] Output: Evaluation results for the best candidates

[1830] Specific operation: The server compares and evaluates multiple candidates based on the adjusted scores. The server selects the candidate with the highest fit.

[1831] Step 21:

[1832] The server generates a report and sends it to the device.

[1833] Input: Evaluation results of the best candidate

[1834] Output: Generated report

[1835] Specific operation: The server automatically generates a report summarizing the evaluation results in detail. The generated report is sent from the server to the terminal so that the user can check it.

[1836] Step 22:

[1837] The user makes the final hiring decision.

[1838] Specific operation: The user checks the report displayed on the terminal and makes a final hiring decision. If necessary, a final interview or additional checks are conducted, and then a formal hiring decision is issued.

[1839] This will streamline the entire hiring process and allow for more accurate and objective evaluations.

[1840] (Application example 2)

[1841] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1842] The recruitment process in a company is extremely important, especially in the security department, and requires accurate assessment of candidates' reliability and suitability. However, this process places a heavy burden on recruiters, making it difficult to secure the right talent. It is also difficult to grasp a candidate's emotional state in real time and make an accurate assessment. To solve these issues, a system is needed to streamline the entire recruitment process and assess candidates' reliability and nervousness.

[1843] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and displaying the emotion analysis results, means for scoring the candidate's match after the interview, means for comparatively evaluating multiple candidates based on the scoring results, means for efficiently securing personnel within the security department, and an emotion engine for evaluating the candidate's reliability and level of tension. This improves the efficiency of the hiring process in the security department and enables the prompt securing of suitable personnel.

[1844] An "enterprise" is a legal entity or organization established to carry out economic activities as an organization and provide goods and services.

[1845] "Recruitment requirements" refer to the standards, such as skills, experience, and qualifications, that a company requires of new employees.

[1846] A "job posting" is a document that provides information such as the type of job and conditions that a company is recruiting for to job seekers.

[1847] A "candidate" is a job seeker who applies for a job at a company and seeks employment.

[1848] "Document" refers to information in written form, such as a resume or curriculum vitae submitted by a candidate.

[1849] "Skills" refer to the knowledge and abilities required to efficiently carry out specific tasks or operations.

[1850] "Experience information" refers to specific information about the job or project a candidate has previously undertaken.

[1851] "Match rate" is an indicator that shows how closely a candidate's skills and experience match the company's hiring requirements.

[1852] "Footage" refers to video data that captures the candidate's appearance and behavior during an interview.

[1853] "Audio" refers to audio data recorded from the candidate's voice during the interview.

[1854] "Emotion analysis results" refer to the evaluation results of the candidate's emotional state obtained by analyzing video and audio data.

[1855] "Scoring" is the process of quantifying and evaluating a candidate's match.

[1856] "Comparative evaluation" is the process of evaluating multiple candidates based on their scoring results and selecting the most suitable candidate.

[1857] "Human resource acquisition" refers to a series of activities that a company undertakes to properly recruit and secure the human resources it needs.

[1858] "Credibility" is an indicator of whether a candidate's words, actions, and behavior are trustworthy.

[1859] "Tension level" is an indicator of the level of stress and anxiety a candidate feels during an interview.

[1860] An "emotion engine" is software or algorithms that analyze video and audio data to assess a candidate's emotional state.

[1861] This invention provides a "security interview assistant" system that enables companies to efficiently and accurately select the personnel they are looking for. The system for realizing this application example will be described in detail below.

[1862] First, the user (company recruiter) logs in to the recruitment portal site and enters the company's recruitment requirements. The entered recruitment requirements are then sent to the server via the terminal. The generative AI model on the server automatically generates a job posting based on these recruitment requirements.

[1863] Candidates upload documents such as their resumes and work histories to a recruitment portal site. These documents are then sent to a server via their device, where the server's natural language processing engine analyzes the documents to extract information about skills and experience. Based on this information, the candidate's match is evaluated.

[1864] When conducting a real-time interview, the user (interviewer) talks with the candidate via a video chat system. The device sends video and audio data to the server in real time. The server's video and audio analysis engines analyze the candidate's facial expressions and tone of voice, and use an emotion engine to assess their level of nervousness and confidence. This allows the interviewer to be presented with suitable question candidates and areas of concern.

[1865] Once the interview is over, the server combines the collected video and audio data, as well as the interviewer's evaluation, and uses an AI model to score each candidate's match. The scoring results are then adjusted, taking into account emotional data, to calculate a final score. Based on the scoring results, the server compares and evaluates multiple candidates, generates a report to select the most suitable candidate, and sends it to the device.

[1866] Specific applications include:

[1867] 1. Automatic job posting generation:

[1868] The server generates the most suitable job posting based on the security-related recruitment requirements entered by users who log in to the recruitment portal site. Example: Automatically generate a job posting for "candidates with specialized knowledge and work experience in network security with at least three years."

[1869] 2. Candidate characteristics analysis and match assessment:

[1870] The server's natural language processing engine analyzes the resumes and CVs uploaded by candidates, extracts security-related skills and experience, and determines the degree of match. For example, keywords such as "CISSP certification" and "security audit experience" are extracted.

[1871] 3. Real-time sentiment analysis during interviews:

[1872] The server analyzes the candidate's video and audio data collected through the video chat system to assess their emotional state in real time. It measures the candidate's level of nervousness and confidence and presents specific questions to the interviewer. For example, it presents questions such as, "Tell us about a security incident you were actually involved in in the past."

[1873] 4. Post-interview scoring and comparative evaluation:

[1874] After the interview, the server integrates the data collected and uses an AI model to quantify and evaluate the candidate's match. A final score is calculated that takes into account emotional data, and multiple candidates are compared and evaluated. For example, Candidate A is scored 90 points, and Candidate B is scored 85 points.

[1875] Real-time analysis using an emotion engine and advanced analysis using generative AI models is expected to streamline the recruitment of talent for security departments and significantly improve corporate recruitment processes.

[1876] Example prompt sentence:

[1877] Analyze and show the candidate's emotional state in real time: Analyze the candidate's nervousness and confidence level. Evaluate the candidate's emotional state in real time and suggest questions that will appear to the interviewer.

[1878] If the candidate seems nervous, suggest questions: for example, about past project experience.

[1879] This allows for the quick and accurate selection of personnel suitable for the security department.

[1880] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1881] Step 1:

[1882] A user logs in to a recruitment portal site. The user uses a terminal to input the company's recruitment requirements. An example of input data is "Network security specialist with at least three years of experience." The input data is sent to the server by the terminal.

[1883] Step 2:

[1884] The server uses a generative AI model to automatically generate a job posting based on the received hiring requirements data. The job posting includes requirements such as "CISSP certification" and "practical experience in network security." The generated job posting is sent from the server to the terminal, where it can be viewed by the user.

[1885] Step 3:

[1886] Candidates upload their resumes and work histories to a recruitment portal site. The terminal sends these document data to a server. An example of input data is "I have experience in security development in the financial industry."

[1887] Step 4:

[1888] The server inputs the received document data into a natural language processing (NLP) engine. The NLP engine analyzes the document and extracts skill and experience information. For example, keywords such as "CISSP certification" and "network security experience" are extracted. The extracted information is stored on the server.

[1889] Step 5:

[1890] The server uses a generative AI model to evaluate the candidate's match based on the extracted skills and experience information. The evaluation results are sent from the server to the device in the form of "high match" or "medium match," and can be viewed by the user.

[1891] Step 6:

[1892] At the start of the interview, the user (interviewer) talks with the candidate through a video call system. The terminal transmits video and audio data to the server in real time. For example, video frames and audio streams are input.

[1893] Step 7:

[1894] The server uses a video analysis engine and an audio analysis engine to analyze the video and audio data in real time. The candidate's facial expressions and tone of voice are extracted from the analysis results. The emotion engine uses this to evaluate the candidate's level of nervousness and confidence. The evaluation results are sent from the server to the terminal in real time, where the interviewer can check them.

[1895] Step 8:

[1896] During the interview, the server generates candidate questions and concerns based on the results of sentiment analysis in real time and displays them on the device. For example, if the candidate is feeling nervous, the server will suggest a question such as, "Tell us about your specific project experience."

[1897] Step 9:

[1898] After the interview, the server combines the collected video and audio data and the interviewer's evaluation. This data is input into an AI model, which scores the candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[1899] Step 10:

[1900] The server adjusts the scoring results based on the emotion data obtained by the emotion engine. After calculating the final score, a comparative evaluation is performed based on multiple candidates to select the most suitable candidate. The selection results are sent from the server to the terminal in report format, allowing the user (company recruiter) to make the final decision.

[1901] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1902] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1903] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1904] [Fourth embodiment]

[1905] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1906] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1907] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1908] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1909] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1910] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1911] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1912] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1913] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1914] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1915] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1916] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1917] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1918] System Overview

[1919] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatically creating job postings, analyzing candidates' characteristics and determining their match, analyzing real-time emotions during interviews, and scoring and comparative evaluation after interviews.

[1920] Program processing flow and specific examples

[1921] Automatic creation of job postings

[1922] 1. A user (company representative) logs in to a recruitment portal site and enters the recruitment requirements. For example, suppose the company is looking for a "software development engineer with at least three years of experience."

[1923] 2. The terminal sends the entered employment requirements to the server.

[1924] 3. The server inputs the received hiring requirements into the AI ​​model, which then generates the optimal job posting for the target and position based on past job postings and hiring history.

[1925] 4. The server sends the generated job posting to the terminal so that the user can check it. After checking, the user can make any necessary corrections and officially publish it.

[1926] Candidate characteristics analysis and match assessment

[1927] 1. The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[1928] 2. The terminal sends the uploaded document to the server.

[1929] 3. The server inputs the received document into a natural language processing (NLP) engine. The NLP engine analyzes the document and extracts skill and experience information. Examples of extracted skills include "financial industry" and "software development experience."

[1930] 4. The server uses the extracted information to evaluate the candidate's match using an AI model. The evaluation results are provided to the user (company representative), who determines whether the candidate has a high match.

[1931] Real-time analysis during the interview

[1932] 1. The user (interviewer) starts the interview and uses the video call system.

[1933] 2. The device transmits the interview video and audio data to the server in real time.

[1934] 3. The server uses a video analysis engine to analyze the candidate's emotions based on their facial expressions and tone of voice, for example, to determine their level of nervousness or confidence.

[1935] 4. The server displays the sentiment analysis results in real time on the interviewer's device and suggests suitable questions and areas of concern. For example, if the candidate is nervous, the server will prompt them to "Tell me more about your specific project experience."

[1936] Post-interview scoring and comparison

[1937] 1. After the interview is completed, the server integrates the collected video and audio data and the interviewer's evaluation.

[1938] 2. The server uses an AI model to score each candidate's match. For example, candidate A might receive 90 points, and candidate B might receive 85 points.

[1939] 3. The server compares candidates based on the scoring results and determines the best candidate.

[1940] 4. The server generates a report containing the comparison results and sends it to the terminal. The user (company representative) makes the final hiring decision based on this report.

[1941] Technical effects

[1942] This system allows companies to streamline their recruitment processes and significantly reduce the burden on managers. In addition, AI-based data analysis enables objective and accurate candidate evaluation, enabling the rapid recruitment of suitable personnel. This is expected to improve a company's competitiveness and optimize the allocation of personnel.

[1943] The processing flow will be explained below.

[1944] Automatic creation of job postings

[1945] Step 1:

[1946] The user (company representative) logs in to the recruitment portal site and enters the recruitment requirements, for example, "Recruiting a software development engineer with at least three years of experience."

[1947] Step 2:

[1948] The terminal transmits the entered employment requirements to the server.

[1949] Step 3:

[1950] The server inputs the received hiring requirements into the AI ​​model, which then retrieves and analyzes past job postings and hiring records from a database.

[1951] Step 4:

[1952] The AI ​​model generates optimal job descriptions, including requirements such as "Java or Python programming experience" and "team development experience."

[1953] Step 5:

[1954] The server sends the generated job posting to the terminal so that the user can check it.

[1955] Step 6:

[1956] After checking the job posting, the user makes any necessary corrections, makes a final check, and then publishes the job posting.

[1957] ---

[1958] Candidate characteristics analysis and match assessment

[1959] Step 1:

[1960] The user (candidate) uploads their resume and application form to the recruitment portal site. For example, they might write, "I have experience in software development in the financial industry."

[1961] Step 2:

[1962] The terminal transmits the uploaded document to the server.

[1963] Step 3:

[1964] The server inputs the received document into a natural language processing (NLP) engine.

[1965] Step 4:

[1966] The NLP engine analyzes documents and extracts skill and experience information, such as keywords like "financial industry" and "software development experience."

[1967] Step 5:

[1968] Based on the extracted information, the server uses an AI model to evaluate the candidate's match, for example, rating them as a "high match."

[1969] Step 6:

[1970] The server sends the evaluation results to the terminal so that the user (company representative) can check them.

[1971] ---

[1972] Real-time analysis during the interview

[1973] Step 1:

[1974] The user (interviewer) starts the interview and uses the video call system.

[1975] Step 2:

[1976] The terminal transmits the interview video and audio data to the server in real time.

[1977] Step 3:

[1978] The server uses a video analysis engine and an audio analysis engine to analyze the candidate's facial expressions and tone of voice in real time.

[1979] Step 4:

[1980] Based on the results of the sentiment analysis, the server displays suitable question candidates and areas of concern on the interviewer's device in real time. For example, if the candidate is nervous, the server will suggest a question such as, "Please tell us more about your specific project experience."

[1981] Step 5:

[1982] The user (interviewer) conducts the interview based on the presented questions and asks additional questions if necessary.

[1983] ---

[1984] Post-interview scoring and comparison

[1985] Step 1:

[1986] The server integrates the collected video and audio data and the interviewer's evaluation after the interview is completed.

[1987] Step 2:

[1988] The server uses an AI model to score each candidate's match, for example, Candidate A might get 90 points, and Candidate B might get 85 points.

[1989] Step 3:

[1990] The server compares and evaluates multiple candidates based on the scoring results and determines the most suitable candidate.

[1991] Step 4:

[1992] The server generates a report containing the comparison results and sends it to the terminal.

[1993] Step 5:

[1994] The user (company representative) makes the final hiring decision based on the received report.

[1995] ---

[1996] Through the above processing steps, the recruitment assistant AI system realizes an efficient and objective recruitment process, significantly reducing the burden on corporate managers.

[1997] Example 1

[1998] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1999] Traditional corporate recruitment processes require a wide range of manual tasks, such as creating job postings, evaluating candidates, conducting interviews, and scoring the results, which is time-consuming and labor-intensive. It is also difficult to analyze candidates' characteristics and emotions in real time, making objective evaluations difficult. Therefore, there is a need to reduce the burden on recruiters and achieve a more efficient and objective recruitment process.

[2000] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2001] In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents and extracting skills and experience information using a natural language processing engine, means for evaluating the candidate's match based on the skills and experience information, means for analyzing video and audio during the interview in real time and displaying the results of a sentiment analysis, means for presenting suitable question candidates and concerns during the interview, means for scoring the candidate's match after the interview, and means for comparatively evaluating multiple candidates based on the scoring results, thereby enabling efficiency improvement and objective evaluation of the entire hiring process.

[2002] "Employment requirements" refer to the specific conditions, such as skills, experience, educational background, and qualifications, that a company seeks in its employees.

[2003] A "job posting" is a document containing recruitment information published by a company, which lists the job types, conditions, and work content.

[2004] "Documents" refers to documents submitted by candidates, such as resumes and application forms.

[2005] A "natural language processing engine" is software or a system that analyzes the content of documents and extracts skill and experience information.

[2006] "Skills and experience information" is information that indicates the candidate's abilities and past work experience.

[2007] "Match" is an evaluation that indicates how closely a candidate's characteristics match the company's hiring requirements.

[2008] "Real-time analysis of video and audio" refers to a technology that analyzes video and audio collected during an interview on the spot.

[2009] "Emotion analysis results" are data showing the emotional state of a candidate obtained by analyzing their facial expressions and tone of voice.

[2010] "Suggested Questions and Concerns" present questions that interviewers should ask candidates and points to be aware of during the interview.

[2011] "Scoring" refers to the process of assigning a score to the interview results and the candidate's characteristics to evaluate them.

[2012] "Comparative evaluation" means comparing multiple candidates based on scoring results.

[2013] System Overview

[2014] This invention relates to a recruitment assistant AI system that streamlines companies' recruitment processes and reduces the burden on managers. This system supports each stage of the recruitment process, including automatically creating job postings, analyzing candidates' characteristics and determining their match, analyzing real-time emotions during interviews, and scoring and comparative evaluation after interviews.

[2015] Automatic creation of job postings

[2016] The user (company representative) accesses the login page using their company's account information and enters specific hiring requirements, such as "software development engineer with at least three years of experience." The device collects the information entered by the user in real time and sends it securely to the server using HTTPS. The server converts the received hiring requirements into an appropriate format and inputs it into an AI model using TensorFlow. The model references a past database and generates an optimal job posting. The server sends the generated job posting in JSON format to the device, which displays it on the user's screen. The user checks the content on the screen, modifies the job posting if necessary, and clicks the "Publish" button, which officially posts the job information on the portal site.

[2017] Examples:

[2018] If a user enters a job requirement for "an engineer with 5+ years of experience in front-end development," the server generates a corresponding job posting and provides it as "React.js developer with 5+ years of experience wanted." The user can then review the generated job posting, modify it, and publish it.

[2019] Example prompt:

[2020] "Please create a job posting for an engineer with at least 5 years of experience in front-end development."

[2021] Candidate characteristics analysis and match assessment

[2022] Users (candidates) upload their resumes and application forms to the system in an appropriate format (PDF or Word). The terminal scans the uploaded files and sends them to the server. The server launches a natural language processing (NLP) engine using PyTorch to analyze the text within the document. For example, it extracts keywords such as "financial industry" and "software development experience." The server inputs the extracted information into a pre-prepared evaluation algorithm and numerically evaluates the match between the candidate's skills and the company's requirements. The evaluation results are sent to the terminal, and the user can view a detailed evaluation of the candidate on the screen.

[2023] Examples:

[2024] If a candidate states that they have "software development experience in the financial industry" in the past, the server will extract "financial industry" and "software development experience" and compare them with the company's job requirements to evaluate the match as 90%.

[2025] Example prompt:

[2026] "Analyze the resumes of candidates with software development experience in the financial industry to determine the degree of match."

[2027] Real-time sentiment analysis during interviews

[2028] The user (interviewer) initiates a video call with a candidate via Zoom or Microsoft Teams. The device captures the video and audio data of the video call in real time and sends it to the server. The server runs a video analysis engine such as OpenCV to analyze the candidate's facial expressions and tone of voice. For example, it analyzes their level of nervousness and confidence. The server then creates candidate questions corresponding to the analysis results and displays them in real time on the interviewer's device. The interviewer can use this information to ask the candidate appropriate questions.

[2029] Examples:

[2030] If sentiment analysis reveals that a candidate is nervous during an interview, the server will display "High nervousness" and suggest questions to the interviewer, such as "Tell me more about your specific project experience."

[2031] Example prompt:

[2032] "Perform real-time sentiment analysis of candidate tension during interviews and suggest appropriate questions."

[2033] Post-interview scoring and comparison

[2034] The server combines the video and audio data collected during the interviews, as well as the interviewers' manual evaluations, and stores them as a single dataset. The server inputs the combined dataset into an AI model, which evaluates each candidate's skills, aptitude, and expressions during the interview to create a score. For example, candidate A may receive a score of 90, while candidate B receives a score of 85. The server compares the scoring results and determines the most suitable candidate, taking into account algorithms and past hiring performance data. The server generates a detailed report including the final evaluation and comparison results and sends it to the device. The user makes the final hiring decision based on this report. The user reviews the report displayed on the device and selects the most suitable candidate. Once the final decision is made, the server sends an offer of employment.

[2035] Technical effects

[2036] This system allows companies to streamline their recruitment processes and significantly reduce the burden on managers. In addition, AI-based data analysis enables objective and accurate candidate evaluation, enabling the rapid recruitment of suitable personnel. This is expected to improve a company's competitiveness and optimize the allocation of personnel.

[2037] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2038] Step 1:

[2039] A user logs in to a recruitment portal site and enters their recruitment requirements.

[2040] Input: User inputs recruitment requirements (e.g., "Software development engineer with at least 3 years of experience")

[2041] What happens: The user logs in using their company's account information and enters their job requirements in the text fields.

[2042] Step 2:

[2043] The terminal transmits the entered employment requirements to the server.

[2044] Input: Recruitment requirements data

[2045] Data processing / calculation: The device collects input information in real time and sends it securely to the server using HTTPS

[2046] Output: Recruitment requirements data sent to the server

[2047] Specific operation: The terminal registers data in the transmission queue and sends it to the server via HTTPS communication.

[2048] Step 3:

[2049] The server inputs the recruitment requirements received into the AI ​​model and generates a job posting.

[2050] Input: Recruitment requirements data received by the server

[2051] Data processing / calculation: The server converts the data into the appropriate format, inputs it into an AI model using TensorFlow, and generates a job posting.

[2052] Output: Generated job posting data

[2053] Specific operation: The server retrieves relevant information from the database and inputs it into the AI ​​model to generate a job posting.

[2054] Step 4:

[2055] The server sends the generated job posting to the terminal and the user confirms it.

[2056] Input: Generated job posting data

[2057] Data processing / calculation: The server converts the generated job posting into JSON format and sends it to the terminal.

[2058] Output: The job posting displayed to the user

[2059] Specific operation: The server sends JSON format data to the terminal, which parses it and displays it on the user's screen.

[2060] Step 5:

[2061] The user modifies and publishes the job posting.

[2062] Input: User-modified job posting data

[2063] Data processing / calculation: The user edits the job posting details on the screen and clicks the "Publish" button.

[2064] Output: Published job postings

[2065] Specific operation: The user checks the content, makes corrections, and then presses the publish button, and the job information is posted on the portal site.

[2066] Step 6:

[2067] The user (candidate) uploads their resume and application form.

[2068] Input: Candidates upload their resumes and application forms

[2069] Data processing / calculation: Upload document data to the system

[2070] Output: Uploaded document data

[2071] What happens: Candidates follow the system instructions and upload documents using the file selection button.

[2072] Step 7:

[2073] The terminal sends the uploaded document to the server.

[2074] Input: Uploaded document data

[2075] Data processing / calculation: The device scans the file and sends it to the server

[2076] Output: Document data sent to the server

[2077] Specific operation: The device temporarily stores the uploaded file and securely sends it to the server.

[2078] Step 8:

[2079] The server inputs the document into an NLP engine for analysis.

[2080] Input: Document data sent to the server

[2081] Data processing / calculation: The server runs a natural language processing (NLP) engine using PyTorch to perform analysis. Specifically, it extracts keywords such as "financial industry" and "software development experience."

[2082] Output: Parsed skills and experience information

[2083] How it works: The server inputs documents into the NLP engine and extracts important skills and experience information.

[2084] Step 9:

[2085] The server uses the analysis results to evaluate the degree of match using an AI model.

[2086] Input: Parsed skills and experience information

[2087] Data processing / calculation: The server inputs the extracted information into an evaluation algorithm and evaluates the candidate's match numerically.

[2088] Output: Evaluation results

[2089] Specific operation: The server scores the extracted information based on an evaluation algorithm.

[2090] Step 10:

[2091] The server provides the evaluation results to the user.

[2092] Input: Evaluation result data

[2093] Data processing / calculation: Evaluation results sent to the terminal

[2094] Output: Evaluation results displayed on the user's screen

[2095] Specific operation: The server sends the evaluation results in JSON format to the terminal, and the user confirms them.

[2096] Step 11:

[2097] The user (interviewer) starts the interview using the video call system.

[2098] Input: Start the video call system

[2099] Data processing / calculation: Capture of interview video and audio data

[2100] Output: Captured video and audio data

[2101] What happens: The interviewer will start a video call using Zoom or Microsoft Teams.

[2102] Step 12:

[2103] The terminal transmits the interview video and audio data to the server.

[2104] Input: Captured video and audio data

[2105] Data processing / calculation: Send to server in real time

[2106] Output: Video and audio data sent to the server

[2107] Specific operation: The device captures video and audio data in real time and sends it to the server.

[2108] Step 13:

[2109] The server performs emotion analysis using a video analysis engine.

[2110] Input: Video and audio data sent to the server

[2111] Data processing / calculation: Emotion analysis is performed using OpenCV etc. to analyze tension and confidence.

[2112] Output: Sentiment analysis results

[2113] Specific operation: The server inputs video and audio data into an analysis engine to evaluate the emotional state.

[2114] Step 14:

[2115] The server displays the analysis results and suitable question candidates on the device.

[2116] Input: Sentiment analysis results

[2117] Data processing / calculation: Generate suitable question candidates and send them to the device

[2118] Output: Question candidates and sentiment analysis results displayed on the user's screen

[2119] Specific operation: The server generates candidate questions based on the analysis results and sends them to the device, which then displays them to the user.

[2120] Step 15:

[2121] The server consolidates the collected data.

[2122] Input: Video and audio data collected during the interview, and manual assessment by the interviewer

[2123] Data processing / calculation: Data integration and storage

[2124] Output: A consolidated dataset

[2125] Specific operation: The server consolidates and centrally manages all data.

[2126] Step 16:

[2127] The server scores each candidate using an AI model.

[2128] Input: Integrated dataset

[2129] Data processing / calculation: Input data into the AI ​​model to generate a score for each candidate (e.g., candidate A scores 90, candidate B scores 85)

[2130] Output: Scoring results

[2131] How it works: The server inputs data into the AI ​​model and generates a score.

[2132] Step 17:

[2133] The server compares the candidates to determine the best candidate.

[2134] Input: Scoring results

[2135] Data processing / calculation: Use comparison algorithms to determine the best candidates

[2136] Output: The best candidate is identified

[2137] Specific operation: The server compares the scoring results and selects the most suitable candidate.

[2138] Step 18:

[2139] The server sends the comparison results as a report to the terminal.

[2140] Input: The best candidate's results

[2141] Data processing / calculation: Generate detailed reports and send them to your device

[2142] Output: The report that is displayed on the user's screen

[2143] Specific operation: The server generates a report based on the evaluation results and sends it to the terminal, which then displays it to the user.

[2144] Step 19:

[2145] The user makes the final hiring decision based on the report.

[2146] Input: Best candidate report

[2147] Data processing / calculation: Making decisions based on reports

[2148] Output: Final hiring decision

[2149] Specific actions: The user reviews the report, selects the best candidates, and sends out job offers.

[2150] (Application example 1)

[2151] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2152] In the recruitment process, analyzing candidate characteristics and emotions during interviews is important, but there is no system that utilizes this data in real time to evaluate risk factors. As a result, companies are forced to rely on subjective judgment when selecting appropriate candidates, making efficient recruitment difficult. In addition, it is difficult to detect security risks early in post-recruitment activities.

[2153] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2154] In this invention, the server includes means for inputting a company's hiring requirements, means for automatically generating a job posting based on the hiring requirements, means for receiving documents submitted by candidates, means for analyzing the documents to extract skill and experience information, means for evaluating the candidate's match based on the skill and experience information, means for analyzing video and audio during the interview in real time and displaying the emotion analysis results, means for analyzing the candidate's behavior in real time during the interview and performing a risk assessment, means for scoring the candidate's match after the interview, and means for comparatively assessing multiple candidates based on the scoring results. This enables a more objective and efficient hiring process based on real-time emotion and risk analysis.

[2155] "Means for inputting a company's recruitment requirements" refers to an interface that allows a company to input the conditions, qualifications, skills, etc. of the personnel it wishes to hire into the system.

[2156] The "means for automatically generating a job posting based on the hiring requirements" refers to an algorithm and system that creates optimal job advertisements and job postings based on the hiring requirements entered.

[2157] "Means for receiving documents submitted by candidates" refers to a function for receiving resumes and application forms uploaded by candidates to the system.

[2158] The "means for analyzing the document and extracting skill and experience information" is a function that uses natural language processing technology to extract information such as the candidate's skills and work history from the received document.

[2159] The "means for evaluating the degree of match of a candidate based on the skills and experience information" refers to an algorithm and system that compares the extracted skills and experience information and evaluates the degree of suitability of a candidate for the recruitment requirements.

[2160] "A means of analyzing video and audio during interviews in real time and displaying the results of emotional analysis" is a function that instantly processes video and audio collected during interviews, analyzes the candidate's facial expressions and tone of voice, and displays their emotional state.

[2161] "Means for analyzing candidate behavior in real time during interviews and conducting risk assessments" refers to a function that observes candidate actions and behaviors occurring during interviews in real time and assesses risk based on that information.

[2162] "Means for scoring candidate match after interview" refers to algorithms and systems that quantify and evaluate a candidate's suitability based on data collected after the interview.

[2163] The "means for comparatively evaluating a plurality of candidates based on the scoring results" is a function for comparing the evaluation results of scored candidates and selecting the most suitable candidate.

[2164] This invention provides a recruitment assistant AI system that integrates multiple functions to streamline a company's recruitment process and reduce the burden on managers. The configuration and operation of this system are described below.

[2165] Automatic creation of job postings

[2166] The server automatically generates job postings based on information received through the company's recruitment requirements input method. At this time, the server uses a generative AI model based on past recruitment data and recruitment performance to cre...

Claims

1. A means of inputting the company's recruitment requirements; A means for automatically generating a job posting based on the hiring requirements; a means of receiving documents submitted by candidates; means for analyzing the document to extract skill and experience information; A means for evaluating the match of candidates based on the skills and experience information; A means to analyze the video and audio during the interview in real time and display the results of the emotion analysis, A way to score candidates' match after the interview is over, A means for comparatively evaluating a plurality of candidates based on the scoring results; A system including:

2. The system of claim 1 further comprising means for performing real-time sentiment analysis during the interview and displaying candidate questions and concerns to the interviewer.

3. The system according to claim 1 , further comprising means for automatically generating a suitability report for the candidate based on the scoring results and providing the report to a recruiter.

Images