System

Abstract

A system is provided.SOLUTION: A terminal for inputting information of an applicant, a server for receiving the information of the applicant transmitted from the terminal, a database connected to the server, the database storing a pass / fail criterion, a system including an AI interviewer connected to the server, the AI interviewer performing a conversation with the applicant, a means for recording and evaluating the conversation, and a means for making a pass / fail determination based on the evaluation and notifying a result to the applicant.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In traditional recruitment interviews for part-time jobs, etc., the store assigns interviewers to conduct the interviews, which requires time and manpower, making it difficult to operate efficiently. Another issue is that the interviewers' subjective judgments make it difficult to ensure fairness and consistency in evaluations. In particular, when there are many applicants, the burden of interviewing each individual is heavy, making it difficult to efficiently hire the right people. Furthermore, since interview data is recorded and analyzed manually, there are also issues with cumbersome information management and utilization. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides an interview system including a terminal for inputting applicant information, a server for receiving the applicant information transmitted from the terminal, a database connected to the server and having means for storing pass / fail criteria, and an AI interviewer connected to the server, having means for engaging in dialogue with the applicant, recording the dialogue, conducting an evaluation, and making a pass / fail decision based on the evaluation and notifying the applicant of the result. This system automates interviews and ensures fairness and consistency in evaluations, reducing the time and effort required by the store. It also enables centralized management of evaluation data, improving the efficiency of the entire hiring process.

[0006] "Terminal" refers to the device on which applicants enter and send information, specifically a PC, smartphone, tablet, etc.

[0007] "Server" refers to a central control device for receiving, storing and processing applicant information sent from the terminals.

[0008] "Database" refers to a system connected to a server for storing and managing applicant information and acceptance criteria.

[0009] "Pass / fail criteria" refers to the evaluation criteria used to determine which applicants pass or fail a job interview.

[0010] "Interview system" refers to a combination of software and hardware that enables dialogue between applicants and AI interviewers.

[0011] An "AI interviewer" is a virtual interviewer that uses artificial intelligence technology to interact with applicants and analyze their responses.

[0012] "Means for recording and evaluating dialogue" refers to the processing function for recording the dialogue with the AI ​​interviewer and then analyzing and evaluating the recorded data.

[0013] "Means for determining whether an applicant has passed or failed and notifying the result" refers to the function of determining whether an applicant has passed or failed based on the evaluation results and notifying the applicant of the result. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention is a recruitment interview system that uses AI, which reduces the time and effort required for job interviews at stores and realizes a fair and efficient recruitment process. The system includes a terminal for inputting applicant information, a server for receiving and processing applicant information, a database for storing pass / fail criteria, an AI interviewer who interacts with applicants, a means for recording and analyzing the interaction, and a means for notifying the results.

[0036] Collection of applicant information

[0037] First, the user (applicant) uses a device (e.g., PC, tablet, smartphone) to enter the necessary information into the application form. This information includes name, available working hours, relevant experience, self-promotion, etc. The information sent from the device is received by the server and stored in a database.

[0038] Setting pass / fail criteria

[0039] The store sets various criteria for acceptance through the management screen, such as communication skills, available working hours, relevant experience, etc. The criteria entered by the store are received by the server and stored in the database.

[0040] Preparing for and conducting interviews

[0041] The server prepares the interview system based on applicant information and pass / fail criteria. The user accesses the system via their device at the designated interview date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue (questions and answers) is recorded in real time by the server. The AI ​​interviewer asks questions based on a pre-set list of questions and accepts the user's answers.

[0042] Evaluation of interview results

[0043] The server analyzes the recorded dialogue logs and evaluates the user's responses. For example, it checks whether the available working hours meet the criteria and whether the relevant experience is appropriate. It also evaluates the user's communication skills using natural language processing. Based on these evaluations, the server calculates an overall score and determines whether the user is accepted or rejected.

[0044] Notification of results

[0045] After the pass / fail decision is complete, the server notifies the applicant of the result, for example by sending an email to the applicant with details of the next steps if they pass. The user receives their result and can confirm the next steps.

[0046] As a specific example, the following flow can be considered.

[0047] 1. Entering applicant information: The user uses a terminal to enter the name "Yamada Taro," available working hours "20 hours per week," related experience "2 years," and self-promotion "I'm good at customer service."

[0048] 2. Receiving and storing: The application information sent from the device is received by the server and stored in a database.

[0049] 3. Setting standards: The store sets the available working hours as "20 hours or more per week," related experience as "1 year or more," and communication skills as "high" on the management screen.

[0050] 4. Conducting the interview: The user logs in to the system on the date and time of the interview and interacts with the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?", and the user replies, "I can work 20 hours per week."

[0051] 5. Evaluation of results: The server analyzes the dialogue log, compares the user's available working hours and related experience with the standard, and evaluates their communication skills using natural language processing.

[0052] 6. Pass / fail determination and notification: The server calculates the applicant's overall score, determines whether they passed or failed, and notifies the user of the result by email.

[0053] The above steps allow stores to evaluate applicants and conduct hiring activities efficiently and fairly, and the system can significantly reduce the time and effort required for interviews.

[0054] The processing flow will be explained below.

[0055] Step 1:

[0056] The store logs in to the management screen and enters the pass / fail criteria (e.g., communication skills, available working hours, related experience, etc.).

[0057] Step 2:

[0058] The terminal transmits the entered pass / fail criteria to the server.

[0059] Step 3:

[0060] The server saves the received pass / fail criteria in the database. After saving is complete, a confirmation message is displayed to the store.

[0061] Step 4:

[0062] The user enters information such as name, available working hours, relevant experience, and self-promotion into the application form.

[0063] Step 5:

[0064] The terminal transmits the input application information to the server.

[0065] Step 6:

[0066] The server receives the application information and saves it in the database. After saving is complete, a confirmation message is displayed to the user confirming the application.

[0067] Step 7:

[0068] The server sets the interview date and time and sends a reminder email to the user.

[0069] Step 8:

[0070] The user logs in to the interview system through a terminal at the designated interview date and time.

[0071] Step 9:

[0072] The server starts the interview system and the AI ​​interviewer starts asking questions based on a pre-set list of questions.

[0073] Step 10:

[0074] The user answers questions posed by the AI ​​interviewer, and the answers are sent to the server in real time and recorded.

[0075] Step 11:

[0076] The server analyzes the recorded dialogue log and evaluates the user's responses.

[0077] Step 12:

[0078] The server calculates the evaluation results based on the pass / fail criteria and determines whether the application passes or fails.

[0079] Step 13:

[0080] The server stores the pass / fail results in a database.

[0081] Step 14:

[0082] The server creates a notification email containing the pass / fail result and sends it to the user.

[0083] Step 15:

[0084] The user receives a notification email and checks the results.

[0085] The above are the detailed process steps from collecting applicant information to interview evaluation and final notification of results. This flow realizes automation and efficiency of interviews.

[0086] Example 1

[0087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0088] The existing recruitment interview process requires a great deal of time and effort, and the interviewer's subjectivity can affect the results. There are multiple problems, such as unfair judgments and reduced efficiency due to busy schedules. Furthermore, inconsistent evaluations of applicants make it difficult to select the best candidates. This invention aims to realize a fair and efficient recruitment process.

[0089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0090] In this invention, the server includes a means for receiving applicant information and storing it in a database, a means for analyzing the recorded dialogue log and evaluating the applicant using natural language processing, and a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result, thereby making it possible to improve the efficiency and fairness of the job interview process.

[0091] "Terminal" means an electronic device used by a user to input and transmit information.

[0092] "Server" means the central system that receives, processes and manages information sent from the terminals.

[0093] A "database" is a storage device within the system that stores and manages data such as applicant information and acceptance criteria.

[0094] The "AI Interviewer" is an artificial intelligence system that interacts with applicants and collects and evaluates their responses based on questions.

[0095] The "interview system" refers to the entire system, including the AI ​​interviewer, that executes and manages the interview process with applicants.

[0096] A "dialogue log" is data that records the content of the conversation between an AI interviewer and an applicant during an interview.

[0097] "Natural language processing" is a technology that allows computers to understand, analyze, and evaluate human language.

[0098] "Pass / fail assessment" is the process of determining whether an applicant passes or fails based on the results of their evaluation.

[0099] "Notification means" refers to the means for informing applicants of the results of their application, such as sending an email.

[0100] In the embodiment for carrying out the invention, this system is a recruitment interview system using AI, which reduces the time and effort required for recruitment interviews on the store side and realizes a fair and efficient recruitment process. This system includes the following elements.

[0101] Collection of applicant information

[0102] The user uses a device (PC, tablet, smartphone, etc.) to open a browser and access the application form. In the application form, they enter information about the applicant, such as their name, available working hours, relevant experience, and self-promotion. The device sends the information entered by the user in JSON format to the server, which then stores the information in a database.

[0103] Setting pass / fail criteria

[0104] The store uses a management terminal to access the management screen and input the pass / fail criteria, such as available working hours, communication skills, relevant experience, etc. The criteria sent from the management screen are received by the server and stored in a database.

[0105] Preparing for and conducting interviews

[0106] The server prepares the interview system based on the applicant's information and pass / fail criteria. At the designated interview date and time, the user accesses the system through their device and begins a dialogue with the AI ​​interviewer. The AI ​​interviewer asks questions from a pre-set list. For example, they may ask, "How many hours per week can you work?" The user responds, and the device sends the answers to the server, which records them in real time.

[0107] Evaluation of interview results

[0108] After the interview, the server analyzes the recorded dialogue log and evaluates the applicant's communication skills using natural language processing. It also checks whether the applicant's answers meet the criteria for acceptance or rejection. For example, it checks whether the applicant's available working hours and related experience meet the criteria. Based on these evaluations, it calculates an overall score and determines whether the applicant is accepted or rejected.

[0109] Notification of results

[0110] After the pass / fail decision is complete, the server notifies the applicant of the result by email. For example, if the applicant passes, an email containing details of the next steps is sent. The user opens the received email, checks the result, and proceeds to the next step.

[0111] For example, consider the following prompt:

[0112] "Please evaluate the applicant's answers based on whether they are available to work 20 hours or more per week, have at least one year of related experience, and have strong communication skills, and calculate an overall score. Based on this score, you will decide whether the applicant has passed or failed, and if successful, you will send an email with details of the next steps."

[0113] The above steps allow stores to evaluate applicants and conduct hiring activities efficiently and fairly, and the system can significantly reduce the time and effort required for interviews.

[0114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0115] Step 1:

[0116] The user displays the input screen. The user opens a browser on a device (PC, tablet, smartphone, etc.) and accesses the application form. Input: Browser URL Output: Application form screen

[0117] Step 2:

[0118] The user enters applicant information. The user enters the name, available working hours, related experience, self-promotion, etc. into the application form. Input: Applicant information (name, available working hours, related experience, self-promotion) Output: Completed application form

[0119] Step 3:

[0120] The terminal sends applicant information to the server. When the user presses the "Send" button, the terminal sends the entered information to the server in JSON format. Input: Applicant information (JSON format) Output: Data sent to the server

[0121] Step 4:

[0122] The server receives the applicant information and saves it in the database. The server saves the received information in the database and returns a response to the terminal indicating that saving has been completed. Input: Applicant information (JSON format) Output: Saved in the database, save completion response

[0123] Step 5:

[0124] The store displays the management screen. The store accesses the management screen using a management terminal. Input: Browser URL Output: Management screen

[0125] Step 6:

[0126] The store enters the pass / fail criteria. From the management screen, the store enters pass / fail criteria such as available working hours, communication skills, and related experience. Input: Available working hours, communication skills, and related experience criteria Output: Entered pass / fail criteria

[0127] Step 7:

[0128] The terminal sends the pass / fail criteria to the server. When the store presses the "Save" button, the terminal sends the criteria to the server in JSON format. Input: Pass / fail criteria (JSON format) Output: Data sent to the server

[0129] Step 8:

[0130] The server saves the pass / fail criteria in the database. The server saves the received criteria in the database and returns a response to the terminal indicating that saving is complete. Input: Pass / fail criteria (JSON format) Output: Saved in the database, save completion response

[0131] Step 9:

[0132] The server prepares the interview system. Based on the applicant information and pass / fail criteria, the server configures the interview system. Input: Applicant information, pass / fail criteria Output: Interview system is ready

[0133] Step 10:

[0134] The user accesses the system at the specified interview date and time. The user logs in to the interview system through a terminal. Input: Interview login URL Output: Interview screen

[0135] Step 11:

[0136] The AI ​​interviewer asks questions. The AI ​​interviewer on the server selects questions from a pre-set list and presents them to the user. Input: Question list Output: Questions asked by the AI ​​interviewer

[0137] Step 12:

[0138] The user answers the question. The user answers the question by text or voice. Input: User's answer Output: Answer data

[0139] Step 13:

[0140] The server records the answers in real time. The server receives the user's answers in real time and saves them as a dialogue log. Input: Answer data Output: Dialogue log

[0141] Step 14:

[0142] The server analyzes the dialogue log. After the interview, the server analyzes the recorded dialogue log and evaluates the applicant's communication skills and responses. Input: Dialogue log Output: Analysis results

[0143] Step 15:

[0144] The server determines whether the candidate passes or fails based on the evaluation results. The server calculates a total score based on criteria such as communication ability, available working hours, and related experience, and determines whether the candidate passes or fails. Input: Analysis results, pass / fail criteria Output: Pass / fail decision

[0145] Step 16:

[0146] The server will notify the result. The server will send the result of the pass / fail decision to the applicant by email. Input: Pass / fail decision result Output: Notification email

[0147] Step 17:

[0148] The user checks the results. The user opens the received email and checks the pass / fail result and next steps. Input: Notification email Output: Check pass / fail result

[0149] (Application example 1)

[0150] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0151] The traditional hiring process had issues such as time-consuming scheduling between applicants and interviewers, and subjective evaluation of applicants. Additionally, conducting interviews and notifying interviewers of results was time-consuming, making efficient hiring difficult. The goal of this invention is to use AI to solve these problems and realize a fairer and more efficient hiring process.

[0152] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0153] In this invention, the server includes a terminal for inputting applicant information, a means for receiving applicant information sent from the terminal, a database connected to the server and a means for storing pass / fail criteria, and an interview system including an AI interviewer connected to the server, which includes a means for engaging in dialogue with the applicant, a means for recording the dialogue between the AI ​​interviewer and the applicant and evaluating them in real time, a means for making a pass / fail decision based on the evaluation and notifying the applicant of the result via an application, and a means for notifying the applicant of the pass / fail decision and providing information on next steps. This automates the job interview process, enabling efficient and fair recruitment activities. Furthermore, using a generative AI model to evaluate applicants improves the reliability and accuracy of the evaluation.

[0154] "Terminal" means the device used by the applicant to enter information, including smartphones, computers, tablets, etc.

[0155] A "server" is a computer system that receives and processes applicant information sent from a terminal.

[0156] "Database" refers to an information storage system for storing and managing applicant information and acceptance criteria.

[0157] "Pass / fail criteria" are criteria used to evaluate applicants during recruitment, and include working hours, related experience, communication skills, etc.

[0158] The "AI Interviewer" is a virtual interviewer that uses artificial intelligence to converse with applicants and analyze and evaluate the content of the conversation.

[0159] "Recording" means saving the conversation between the AI ​​interviewer and the applicant in digital format.

[0160] "Real-time" means that processing and evaluation occurs immediately during the conversation with the applicant.

[0161] "Evaluation" is the act of judging an applicant's suitability and ability based on the applicant's answers and the content of the dialogue.

[0162] "Pass / fail decision" refers to determining whether an applicant meets the employment criteria based on the evaluation results.

[0163] "Notification of results" refers to the act of informing applicants of the results of the pass / fail decision.

[0164] An "application" is software that runs on a smartphone or other digital device and is used by applicants to receive results or conduct interviews.

[0165] "Procedural information" refers to specific information provided to successful applicants as the next step, including information on required documents and start dates.

[0166] A "generative AI model" is an artificial intelligence model trained to perform natural language processing and data analysis, and is used to analyze and evaluate applicant responses.

[0167] The present invention is a system for automating the job interview process. This system is designed to perform an entire process from inputting applicant information to notifying the interview results. Detailed embodiments of the present invention are described below.

[0168] Hardware and software used

[0169] 1. Device: The device used by the applicant to enter information, including smartphones, computers, tablets, etc. In particular, we will develop cross-platform mobile applications using React Native.

[0170] 2. Server: This is a computer system that receives and processes applicant information sent from the terminal. The server is built using Node.js and Express and performs data processing.

[0171] 3. Database: An information storage system for storing and managing applicant information and acceptance criteria. MySQL (registered trademark) is used.

[0172] 4. AI Interviewer: An artificial intelligence that interacts with applicants and uses generative AI models such as GPT-4 (registered trademark) to process interview questions and applicant responses.

[0173] 5. Natural language processing system: Natural language processing is implemented using Python to analyze the applicants' responses.

[0174] 6. Application: This is the software that allows applicants to go through the interview process and notify them of the evaluation results. Applicants participate in the interview and receive the results via the application.

[0175] Data processing and calculation

[0176] The server processes and calculates the data as follows:

[0177] 1. The applicant uses a terminal to enter information such as name, available working hours, relevant experience, and self-promotion into the application. This information is then sent to the server via a REST API.

[0178] 2. The server receives this information and stores it in a MySQL database, where each applicant is assigned a unique ID.

[0179] 3. At the designated interview date and time, the applicant accesses the system through the application and begins a dialogue with the AI ​​interviewer. The AI ​​interviewer asks the applicant questions based on a pre-set list of questions and accepts the applicant's answers.

[0180] 4. The AI ​​interviewer (generative AI model) analyzes the applicant's responses in real time and generates a score for evaluation. For example, it checks whether the applicant's available hours and relevant experience meet the set criteria. Communication skills are also evaluated using a natural language processing system.

[0181] 5. The server makes a comprehensive pass / fail decision based on the analyzed dialogue log. The result of this decision is notified to the application used by the applicant. Successful applicants are then informed of the next steps.

[0182] Specific examples

[0183] As a concrete example, let's say an applicant enters the name "Yamada Taro," his available working hours as "20 hours per week," his related experience as "2 years," and his self-promotion as "I'm good at customer service." This information is sent to the server and saved in the database.

[0184] At the time of the interview, the applicant logs in through the application and speaks to the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?" and the applicant replies, "I can work 20 hours per week." This response is recorded in real time and analyzed by GPT-4. Based on the analysis results, the server makes a pass / fail decision and notifies the applicant of the result.

[0185] Prompt Sentence Examples

[0186] Here are some examples of prompts for generative AI models:

[0187] An applicant named "Yamada Taro" exists in the recruitment interview system. His available working hours are "20 hours per week," his related experience is "2 years," and his self-promotional statement is "I'm good at customer service." Based on this information, his communication skills are evaluated according to the following criteria, and a pass / fail decision is made.

[0188] Available working hours: 20 hours or more per week

[0189] Related experience: 1+ years

[0190] Communication skills: High

[0191] Question list:

[0192] 1. "How many hours per week can you work?"

[0193] 2. "What is your previous work experience like?"

[0194] 3. "Please tell us about yourself."

[0195] Based on these questions, analyze Yamada Taro's answers and determine whether he passed or failed.

[0196] Although a detailed description of the present invention has been given, it will be apparent to those skilled in the art that various changes and modifications can be made thereto without departing from the spirit and scope of the invention.

[0197] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0198] Step 1:

[0199] Input: Applicants use a terminal to enter information such as their name, available hours, relevant experience, and personal information into the application.

[0200] How it works: Data entered on the terminal is sent to the server via a REST API through a front-end form.

[0201] Output: The server stores the received applicant information in a database. Each applicant is assigned a unique ID.

[0202] Step 2:

[0203] Input: The server receives the applicant information sent from the terminal.

[0204] How it works: The server uses Node.js and Express to structure the data it receives and store it in a MySQL database.

[0205] Output: The saved data can be accessed from the management screen, and applicant information can be managed centrally.

[0206] Step 3:

[0207] Input: Receive notifications about interview dates and times.

[0208] How it works: The server uses the calendar functionality within the application to send interview date and time reminders to the applicant.

[0209] Output: The applicant receives a notification and accesses the interview system at the specified date and time.

[0210] Step 4:

[0211] Input: The applicant accesses the system at the designated interview date and time and begins a dialogue with the AI ​​interviewer.

[0212] How it works: An AI interviewer (GPT-4) asks applicants questions based on a pre-defined list, and their answers are recorded in real time.

[0213] Output: The applicant's answers are immediately received by the AI ​​interviewer and sent to the server for analysis.

[0214] Step 5:

[0215] Input: The server receives the applicant's response data received from the AI ​​interviewer.

[0216] How it works: The server uses Python for natural language processing to analyze applicants' responses, evaluate their communication skills and other criteria, and calculate an overall score.

[0217] Output: Scores and analysis results are stored in a database.

[0218] Step 6:

[0219] Input: The analysis results and overall score are saved on the server.

[0220] Operation: The server determines whether the applicant passes or fails based on this data, and generates and saves the next procedure information for those who pass.

[0221] Output: A pass / fail result is prepared.

[0222] Step 7:

[0223] Input: The pass / fail result is saved on the server.

[0224] Operation: The server sends the result of the application to the applicant via the notification system. The result details can also be viewed within the applicant's application.

[0225] Output: Applicants receive their results through the application and receive instructions on next steps.

[0226] Example prompt sentence:

[0227] An applicant named "Yamada Taro" exists in the recruitment interview system. His available working hours are "20 hours per week," his related experience is "2 years," and his self-promotional statement is "I'm good at customer service." Based on this information, his communication skills are evaluated according to the following criteria, and a pass / fail decision is made.

[0228] Available working hours: 20 hours or more per week

[0229] Related experience: 1+ years

[0230] Communication skills: High

[0231] Question list:

[0232] 1. "How many hours per week can you work?"

[0233] 2. "What is your previous work experience like?"

[0234] 3. "Please tell us about yourself."

[0235] Based on these questions, analyze Yamada Taro's answers and determine whether he passed or failed.

[0236] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0237] The present invention combines an emotion engine with an AI-based job interview system to further improve the fairness, impartiality, and accuracy of interview evaluations. This system includes a terminal for inputting applicant information, a server for receiving and processing applicant information, a database for storing pass / fail criteria, an AI interviewer who interacts with applicants, a means for recording and analyzing the interaction, and a means for notifying the results. The emotion engine of the present invention also analyzes the user's emotional state and reflects it in the interview questions and evaluations.

[0238] Collection of applicant information

[0239] First, the user (applicant) uses a device (e.g., PC, tablet, smartphone) to enter information such as their name, available working hours, relevant experience, self-promotion, etc. The information sent from the device is received by the server and stored in a database.

[0240] Setting pass / fail criteria

[0241] The store sets the criteria for acceptance through the management screen, including communication skills, available working hours, relevant experience, etc. The criteria entered by the store is received by the server and stored in the database.

[0242] Preparing for and conducting interviews

[0243] The server prepares the interview system based on applicant information and pass / fail criteria. The user accesses the system via their device at the designated interview date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue (questions and answers) is recorded in real time by the server. The AI ​​interviewer asks questions based on a pre-set list of questions, and the user answers them.

[0244] Emotion Engine Operation

[0245] The emotion engine connected to the server analyzes the user's voice tone, facial expressions, and language patterns to determine their emotional state. For example, it detects whether the user's voice sounds tense or relaxed. Based on the user's emotional state, the AI ​​interviewer dynamically adjusts the questions and responses it asks. For example, if the user is nervous, it can add questions to relax them.

[0246] Evaluation of interview results

[0247] The server evaluates the user's responses based on the recorded dialogue log and the results of emotion analysis by the emotion engine. It checks whether the available working hours meet the criteria and whether the relevant experience is appropriate. It evaluates communication skills using natural language processing, and also takes into account the results of emotion analysis by the emotion engine. This calculates an overall score and determines whether the candidate passes or fails.

[0248] Notification of results

[0249] After the pass / fail decision is complete, the server notifies the applicant of the result, for example by sending an email to the user with detailed instructions on what to do if they pass. The user receives their result and confirms the next steps.

[0250] As a specific example, the following flow can be considered.

[0251] 1. Entering applicant information: The user uses a terminal to enter the name "Yamada Taro," available working hours "20 hours per week," related experience "2 years," and self-promotion "I'm good at customer service."

[0252] 2. Receiving and storing: The application information sent from the device is received by the server and stored in a database.

[0253] 3. Setting standards: The store sets the available working hours as "20 hours or more per week," related experience as "1 year or more," and communication skills as "high" on the management screen.

[0254] 4. Conducting the interview: The user logs in to the system on the date and time of the interview and interacts with the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?", and the user replies, "I can work 20 hours per week."

[0255] 5. Emotion analysis: The server's emotion engine detects nervousness from the user's tone of voice and facial expressions, and changes the AI ​​interviewer's questions to make them more relaxed.

[0256] 6. Evaluation of results: The server evaluates the dialogue log and the emotion analysis results, calculates an overall score, and determines whether the user passes or fails.

[0257] 7. Notification of result: The server creates a success notification email and sends it to the user. The user receives the success notification and confirms the next steps.

[0258] As described above, the present invention further promotes automation and efficiency of interviews, and enables more precise evaluation and response that takes into account the user's emotional state. This allows stores to conduct efficient and fair recruitment activities.

[0259] The processing flow will be explained below.

[0260] Step 1:

[0261] The store logs in to the management screen and enters the pass / fail criteria (e.g., communication skills, available working hours, related experience, etc.).

[0262] Step 2:

[0263] The terminal transmits the entered pass / fail criteria to the server.

[0264] Step 3:

[0265] The server saves the received pass / fail criteria in the database. After saving is complete, a confirmation message is displayed to the store.

[0266] Step 4:

[0267] The user enters information such as name, available working hours, relevant experience, and self-promotion into the application form.

[0268] Step 5:

[0269] The terminal transmits the input application information to the server.

[0270] Step 6:

[0271] The server receives the application information and saves it in the database. After saving is complete, a confirmation message is displayed to the user confirming the application.

[0272] Step 7:

[0273] The server sets the interview date and time and sends a reminder email to the user.

[0274] Step 8:

[0275] The user logs in to the interview system through a terminal at the designated interview date and time.

[0276] Step 9:

[0277] The server starts the interview system and the AI ​​interviewer starts asking questions based on a pre-set list of questions.

[0278] Step 10:

[0279] The user answers questions posed by the AI ​​interviewer, and the answers are sent to the server in real time and recorded.

[0280] Step 11:

[0281] An emotion engine connected to the server analyzes the user's voice tone, facial expressions, and language patterns. For example, if a user answers, "I'm a little nervous, but I want to try this job," the emotion engine will detect nervousness from the user's voice tone and facial expressions.

[0282] Step 12:

[0283] The emotion engine determines the user's emotional state based on the analysis results, generating a result such as "I'm nervous."

[0284] Step 13:

[0285] The server receives the results of the sentiment analysis and dynamically adjusts the questions asked by the AI ​​interviewer, for example, changing the question to, "Let's relax a bit. What has been the most rewarding experience you've had so far?"

[0286] Step 14:

[0287] The user answers a new question, and this answer is also sent in real time to the server and recorded.

[0288] Step 15:

[0289] The server evaluates the user's responses based on the dialogue log and the results of emotion analysis by the emotion engine, for example, checking whether the available working hours meet the criteria and whether the relevant experience is appropriate.

[0290] Step 16:

[0291] The server uses natural language processing to evaluate communication skills, and then incorporates the results of the emotion engine analysis into the evaluation to calculate an overall score.

[0292] Step 17:

[0293] The server determines whether the application passes or fails based on the pass / fail criteria.

[0294] Step 18:

[0295] The server stores the pass / fail results in a database.

[0296] Step 19:

[0297] The server creates a notification email containing the pass / fail result and sends it to the user.

[0298] Step 20:

[0299] The user receives a notification email confirming the result, for example, "Congratulations, you passed!" along with instructions on what to do next.

[0300] The above are the detailed processing steps that combine the emotion engine to collect applicant information, evaluate the interview, and notify the final result. This flow realizes automation and efficiency of interviews, and enables more precise evaluation and response that takes into account the user's emotional state.

[0301] Example 2

[0302] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0303] Conventional recruitment interview systems are often influenced by the interviewer's subjectivity and emotions, which can compromise fairness and impartiality. Furthermore, there is a lack of means to properly grasp the applicant's emotional state, which can lead to a decline in the quality of responses and questions during the interview process. Furthermore, there is no system in place to improve the accuracy of evaluations, resulting in a lack of reliability in hiring decisions. There is a need to solve these problems and conduct fair and efficient recruitment activities.

[0304] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a terminal for inputting applicant information, means for receiving applicant information transmitted from the terminal, a database connected to the server and means for storing pass / fail criteria, an AI system connected to the server and means for engaging in a dialogue with the applicant, and an emotion analysis device connected to the server and means for analyzing the user's emotional state, means for recording the dialogue and making an evaluation, and means for making a pass / fail decision based on the evaluation and notifying the applicant of the result. This enables fair and reliable hiring decisions that take into account the applicant's emotional state.

[0305] A "terminal for inputting applicant information" is a device that allows a user to input their own personal information and job-related data, and includes a PC, tablet, smartphone, etc.

[0306] A "server" is a computer system that receives and sends data over a network and manages and controls the operation of databases and various applications.

[0307] "Database" means a system for efficiently managing and accessing recorded information, including the means for storing applicant information and acceptance criteria.

[0308] "Means for storing acceptance criteria" refers to a data storage system for recording and managing the criteria for employment, including criteria such as available working hours and relevant experience.

[0309] An "AI system" is a system that uses artificial intelligence technology to automatically perform specific tasks and analyses, and includes a means for interacting with applicants.

[0310] An "emotion analysis device" is a device that analyzes a user's voice tone, facial expressions, language patterns, etc. to determine their emotional state.

[0311] "Means for recording and evaluating the dialogue" refers to software or hardware for recording the dialogue between the applicant and the AI ​​system and evaluating that content.

[0312] The "means for determining whether an applicant passes or fails and notifying the applicant of the result" is a system for determining whether an applicant passes or fails based on the content of the dialogue and the evaluation results, and notifying the applicant of the result.

[0313] This invention is a system that improves fairness and evaluation accuracy by combining emotion analysis technology with an AI-based recruitment interview system. This system consists of the following main hardware and software:

[0314] System Configuration

[0315] 1. Terminal: A device on which users enter their application information and interact with the AI ​​interviewer. Terminals include PCs, tablets, smartphones, etc.

[0316] 2. Server: A central computer system that receives, processes, and stores application information, and also integrates and manages AI systems and emotion analysis devices.

[0317] 3. Database: A data storage system for storing applicant information and acceptance criteria.

[0318] 4. AI System: An artificial intelligence-based system for interacting with applicants.

[0319] 5. Emotion analysis device: A device that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[0320] Collection of applicant information

[0321] The user uses the terminal to input information such as name, available working hours, related experience, and self-promotion. For example, the user might input "Yamada Taro," "Available 20 hours per week," "2 years of related experience," and "I'm good at customer service." The input information is sent from the terminal to the server and stored in a database.

[0322] Setting pass / fail criteria

[0323] The store sets the pass / fail criteria for the available working hours, related experience, communication skills, etc. through the management screen. For example, they can set "20 hours or more per week," "1 year or more of related experience," and "high communication skills." These criteria are sent from the terminal to the server and stored in a database.

[0324] Preparing for and conducting interviews

[0325] The server prepares the interview system based on the applicant information and pass / fail criteria. The user accesses the system from their device at the specified date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue is recorded on the server in real time. For example, the AI ​​interviewer asks, "How many hours per week can you work?" and the user replies, "I can work 20 hours per week."

[0326] Emotion analysis

[0327] The emotion analyzer analyzes the user's tone of voice and facial expressions to determine their emotional state. For example, if the AI ​​interviewer detects that the user is nervous, it will add questions to help them relax. It dynamically changes the questions to include, "Is there anything you can do to relax?"

[0328] Evaluation of interview results

[0329] The server comprehensively evaluates the user's responses based on the recorded dialogue log and the results of sentiment analysis. It checks whether the user's available working hours and related experience meet the criteria, and evaluates their communication skills using natural language processing. For example, a user with strong communication skills will respond calmly and clearly.

[0330] Notification of results

[0331] After the pass / fail decision is complete, the server notifies the applicant of the result. For example, if the applicant passes, an email is sent to the user saying, "Congratulations! Here are the detailed steps to proceed to the next step." The user receives the notification and confirms the next steps.

[0332] This system automates the hiring process and enables precise evaluations that take into account the emotional state of applicants, allowing stores to conduct hiring activities efficiently and fairly.

[0333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0334] Step 1:

[0335] The user uses a terminal to input application information. The input information includes name, available working hours, related experience, self-promotion, etc. For example, the user might input "Yamada Taro," "Available 20 hours per week," "2 years of related experience," and "I'm good at customer service." The input data is temporarily saved on the terminal.

[0336] Input: Application information entered by the user into the device

[0337] Output: Application information temporarily saved on the device

[0338] Step 2:

[0339] The terminal sends the application information to the server. The terminal formats the input data and sends it to the server in the form of an HTTP request or similar. The server receives this request.

[0340] Input: Application information temporarily saved on your device

[0341] Output: Application information sent to the server

[0342] Step 3:

[0343] The server stores the received application information in a database. During this process, the server uses a database management system to check the integrity of the data and store the applicant information securely.

[0344] Input: Application information sent to the server

[0345] Output: Application information stored in a database

[0346] Step 4:

[0347] The store sets the pass / fail criteria through the management screen. For example, they can input criteria such as available working hours of "20 hours or more per week," related experience of "1 year or more," and communication skills of "high." The input criteria are sent from the terminal to the server.

[0348] Input: Pass / fail criteria entered by the store into the terminal

[0349] Output: Pass / fail criteria sent to the server

[0350] Step 5:

[0351] The server stores the received pass / fail criteria in a database, at which point the server checks the integrity of the data and adds the criteria to the database.

[0352] Input: Pass / fail criteria sent to the server

[0353] Output: Pass / fail criteria stored in the database

[0354] Step 6:

[0355] As the interview date and time approaches, the server configures the interview system based on the applicant's information and the criteria for acceptance or rejection, including the list of questions for the AI ​​interviewer and the interview schedule.

[0356] Input: Application information and acceptance criteria stored in the database

[0357] Output: Interview system settings

[0358] Step 7:

[0359] The user accesses the system through a terminal at the specified date and time and begins a conversation with the AI ​​interviewer. The terminal sends a connection request to the server, and the server launches the AI ​​interviewer.

[0360] Input: User access request

[0361] Output: Launched AI interviewer

[0362] Step 8:

[0363] The AI ​​interviewer asks questions to the applicant, and the user answers. This dialogue is sent to the server in real time. For example, the AI ​​interviewer asks, "How many hours can you work per week?" and the user answers, "I can work 20 hours per week."

[0364] Input: User's answer

[0365] Output: Real-time conversation logs recorded on the server

[0366] Step 9:

[0367] The emotion analyzer analyzes the user's tone of voice and facial expressions to determine their emotional state. For example, if the server detects that the user is nervous, it will dynamically change the AI ​​interviewer's questions to make them more relaxed.

[0368] Input: User's voice tone and facial expression data

[0369] Output: Emotional state analysis result

[0370] Step 10:

[0371] The server evaluates the user's responses based on the recorded dialogue logs and sentiment analysis results. This evaluation involves determining whether the user's available working hours and relevant experience meet the criteria, and using natural language processing to evaluate their communication skills.

[0372] Input: Dialogue logs and sentiment analysis results

[0373] Output: Evaluation results and overall score

[0374] Step 11:

[0375] After the pass / fail decision is complete, the server notifies the applicant of the result. For example, if the applicant passes, an email is sent to the user saying, "Congratulations! Here are the detailed steps to proceed to the next step." The user can then check the result on their device.

[0376] Input: Evaluation results and overall score

[0377] Output: Result email sent to applicant

[0378] (Application example 2)

[0379] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] Conventional recruitment interview systems did not take into account the emotional state of applicants during evaluation, resulting in problems with the fairness and impartiality of the interview. Furthermore, they were unable to analyze the emotional state of passengers in real time and optimize the in-car environment accordingly. This resulted in passengers' riding experiences not always being satisfactory. Therefore, the challenge is to provide a recruitment interview system that enables emotional analysis of applicants and provides fair and precise evaluations, as well as a system that provides a comfortable riding experience that reflects the emotional state of passengers.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0382] In this invention, the server includes a terminal for inputting applicant information, a server that receives the applicant information transmitted from the terminal, a database connected to the server and having means for storing pass / fail criteria, and an interview system including an AI interviewer connected to the server and having means for engaging in a dialogue with the applicant, means for recording the dialogue and making an evaluation, means for analyzing the emotional state of passengers and dynamically adjusting the in-car environment, and means for making a pass / fail decision based on the evaluation and notifying the applicant of the result. This enables fair and precise evaluation that takes into account the emotional state of the applicant, and also enables the in-car environment to be optimized based on the emotional state of passengers, resulting in a more satisfying riding experience.

[0383] A "terminal" is a device through which applicants or customers enter information.

[0384] "Server" means a central computer group that receives and processes information sent from the Terminals.

[0385] "Database" means a storage device for storing pass / fail criteria and other relevant information.

[0386] An "AI interviewer" is software based on artificial intelligence that is used to interact with applicants.

[0387] The "interview system" is a set of mechanisms that includes an AI interviewer to interact with applicants, record the interaction, and evaluate it.

[0388] "Means for conducting evaluation" means a method or device for evaluating the abilities and qualifications of an applicant based on the dialogue records or other criteria.

[0389] The "means for determining whether an applicant passes or fails and notifying the applicant of the result" refers to a method or device for determining whether an applicant passes or fails based on the evaluation and notifying the applicant of the result.

[0390] The "means for analyzing emotional state" refers to a method or apparatus that analyzes voice tone and facial expressions to determine the user's emotions.

[0391] The "means for dynamically adjusting the in-car environment" refers to a method or device that automatically changes the background music, temperature, lighting, seat angle, etc. in the car based on the user's emotional state.

[0392] "Riding customers" are people riding in the autonomous vehicle.

[0393] To implement this invention, a terminal for inputting applicant information is first prepared. The terminal can be a smartphone, tablet, or PC. The applicant uses the terminal to input information such as their name, available working hours, relevant experience, and self-promotion. The input information is sent to a server, which receives it and stores it in a database.

[0394] Next, the server receives the pass / fail criteria from the administrator and stores them in the database. The administrator sets the criteria such as available working hours, relevant experience, communication ability, etc. The pass / fail criteria are important indicators used to evaluate the job interview.

[0395] The server prepares an AI interviewer based on applicant information and the pass / fail criteria. The AI ​​interviewer uses natural language processing technology to converse with the applicant. During the conversation, the server activates an emotion engine that analyzes the applicant's tone of voice and facial expressions to understand the applicant's emotional state. The emotion engine is built using software such as TENSORFLOW (registered trademark) and OpenCV. For example, it can add questions to relax nervous applicants.

[0396] After the conversation is over, the server evaluates the applicant based on the content of the conversation and the results of sentiment analysis. Using natural language processing technology, the server evaluates the applicant's communication skills and calculates an overall score based on criteria such as available working hours and relevant experience. Based on this, the server determines whether the applicant is successful and notifies the applicant of the result.

[0397] The system also has the functionality to be used in autonomous vehicles. It can analyze the emotional state of passengers and dynamically adjust the in-car environment. Using the smartphone's camera and microphone, it analyzes the passenger's facial expressions and tone of voice and automatically adjusts the in-car background music, temperature, lighting, seat angle, and more, providing the optimal experience for passengers while they are in the vehicle.

[0398] Examples:

[0399] After a user gets into an autonomous vehicle, they launch the "Emotion Reader" app. The app uses the smartphone's camera and microphone to analyze the user's emotional state in real time. For example, if the app detects that the user's facial expression indicates "surprise" or "tension," it automatically adjusts the in-car background music to relaxing music. Furthermore, after a long ride, the app reports any changes in the user's emotions (from tension to relaxation).

[0400] Hardware and software used:

[0401] Smartphone: A device equipped with a camera and microphone to analyze the emotional state of passengers.

[0402] Python: The primary programming language used.

[0403] OpenCV: Used for real-time analysis of camera footage.

[0404] TensorFlow: Used to build the sentiment analysis model.

[0405] Firebase: Used for database and real-time updates.

[0406] Flask: A microframework for providing a sentiment analysis API on the backend.

[0407] Example prompt sentence:

[0408] "Tell me about a system in which a smartphone app inside a self-driving vehicle uses cameras and microphones to analyze passenger emotions in real time and optimize the ride experience (background music, temperature, lighting)."

[0409] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0410] Step 1:

[0411] Input: The applicant enters their name, available working hours, relevant experience, and self-promotional information into the terminal.

[0412] Output: The entered information is sent to the server.

[0413] Specific operation: The user enters the required information into the form on the device and presses the submit button. The device converts the entered information into JSON format and sends it to the server. The server saves the received information in a database in real time.

[0414] Step 2:

[0415] Input: The administrator inputs the pass / fail criteria from the terminal.

[0416] Output: The pass / fail criteria is sent to the server and stored in a database.

[0417] Specific operation: The administrator sets criteria such as available working hours, relevant experience, and communication skills on the management screen and sends them to the server, which receives them and stores them in the database.

[0418] Step 3:

[0419] Input: The server obtains applicant information and acceptance criteria.

[0420] Output: The AI ​​interviewer is prepared.

[0421] Specific operation: The server retrieves applicant information and pass / fail criteria from the database, and prepares an AI interviewer based on that information. The AI ​​interviewer loads a pre-set list of questions.

[0422] Step 4:

[0423] Input: The applicant uses the terminal to access the interview system.

[0424] Output: A conversation between the AI ​​interviewer and the applicant begins.

[0425] Specific operation: The applicant logs in to the system from their terminal at the designated interview date and time. The server authenticates the applicant and starts a dialogue session with the AI ​​interviewer. The dialogue is recorded in real time.

[0426] Step 5:

[0427] Input: AI interviewer analyzes applicant's tone of voice and facial expressions.

[0428] Output: Sentiment analysis results are obtained.

[0429] Specific operation: The server analyzes the camera footage using OpenCV to recognize the applicant's facial expressions, and analyzes the voice data using a TensorFlow model to analyze the voice tone. The analysis results are saved on the server.

[0430] Step 6:

[0431] Input: Dialogue records and sentiment analysis results are collected on the server.

[0432] Output: The applicant's overall rating is calculated.

[0433] How it works: The server uses natural language processing technology to evaluate the applicant's communication skills based on the dialogue log and sentiment analysis results, and calculates an overall score based on criteria such as available working hours and related experience.

[0434] Step 7:

[0435] Input: The server will determine whether the candidate passes or fails based on the overall evaluation.

[0436] Output: The applicant is notified of the results.

[0437] Specific operation: The server determines whether the applicant has passed or failed based on the overall evaluation score, and notifies the applicant of the result by email. The applicant receives the result and confirms the next steps.

[0438] Step 8:

[0439] Input: A customer enters an autonomous vehicle and launches a smartphone app.

[0440] Output: The in-car environment is dynamically adjusted based on the customer's emotional state.

[0441] How it works: The user launches the "Emotion Reader" smartphone app while in the car. The smartphone's camera and microphone are used to analyze the customer's facial expressions and tone of voice in real time. Based on the analysis results, the car's background music, temperature, lighting, seat angle, and other settings are automatically adjusted.

[0442] Example prompt sentence:

[0443] "Tell me about a system in which a smartphone app inside a self-driving vehicle uses cameras and microphones to analyze passenger emotions in real time and optimize the ride experience (background music, temperature, lighting)."

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0445] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0446] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0447] [Second embodiment]

[0448] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0449] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0450] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0451] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0452] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0454] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0455] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0456] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0457] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0458] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0459] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0460] This invention is a recruitment interview system that uses AI, which reduces the time and effort required for job interviews at stores and realizes a fair and efficient recruitment process. The system includes a terminal for inputting applicant information, a server for receiving and processing applicant information, a database for storing pass / fail criteria, an AI interviewer who interacts with applicants, a means for recording and analyzing the interaction, and a means for notifying the results.

[0461] Collection of applicant information

[0462] First, the user (applicant) uses a device (e.g., PC, tablet, smartphone) to enter the necessary information into the application form. This information includes name, available working hours, relevant experience, self-promotion, etc. The information sent from the device is received by the server and stored in a database.

[0463] Setting pass / fail criteria

[0464] The store sets various criteria for acceptance through the management screen, such as communication skills, available working hours, relevant experience, etc. The criteria entered by the store are received by the server and stored in the database.

[0465] Preparing for and conducting interviews

[0466] The server prepares the interview system based on applicant information and pass / fail criteria. The user accesses the system via their device at the designated interview date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue (questions and answers) is recorded in real time by the server. The AI ​​interviewer asks questions based on a pre-set list of questions and accepts the user's answers.

[0467] Evaluation of interview results

[0468] The server analyzes the recorded dialogue logs and evaluates the user's responses. For example, it checks whether the available working hours meet the criteria and whether the relevant experience is appropriate. It also evaluates the user's communication skills using natural language processing. Based on these evaluations, the server calculates an overall score and determines whether the user is accepted or rejected.

[0469] Notification of results

[0470] After the pass / fail decision is complete, the server notifies the applicant of the result, for example by sending an email to the applicant with details of the next steps if they pass. The user receives their result and can confirm the next steps.

[0471] As a specific example, the following flow can be considered.

[0472] 1. Entering applicant information: The user uses a terminal to enter the name "Yamada Taro," available working hours "20 hours per week," related experience "2 years," and self-promotion "I'm good at customer service."

[0473] 2. Receiving and storing: The application information sent from the device is received by the server and stored in a database.

[0474] 3. Setting standards: The store sets the available working hours as "20 hours or more per week," related experience as "1 year or more," and communication skills as "high" on the management screen.

[0475] 4. Conducting the interview: The user logs in to the system on the date and time of the interview and interacts with the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?", and the user replies, "I can work 20 hours per week."

[0476] 5. Evaluation of results: The server analyzes the dialogue log, compares the user's available working hours and related experience with the standard, and evaluates their communication skills using natural language processing.

[0477] 6. Pass / fail determination and notification: The server calculates the applicant's overall score, determines whether they passed or failed, and notifies the user of the result by email.

[0478] The above steps allow stores to evaluate applicants and conduct hiring activities efficiently and fairly, and the system can significantly reduce the time and effort required for interviews.

[0479] The processing flow will be explained below.

[0480] Step 1:

[0481] The store logs in to the management screen and enters the pass / fail criteria (e.g., communication skills, available working hours, related experience, etc.).

[0482] Step 2:

[0483] The terminal transmits the entered pass / fail criteria to the server.

[0484] Step 3:

[0485] The server saves the received pass / fail criteria in the database. After saving is complete, a confirmation message is displayed to the store.

[0486] Step 4:

[0487] The user enters information such as name, available working hours, relevant experience, and self-promotion into the application form.

[0488] Step 5:

[0489] The terminal transmits the input application information to the server.

[0490] Step 6:

[0491] The server receives the application information and saves it in the database. After saving is complete, a confirmation message is displayed to the user confirming the application.

[0492] Step 7:

[0493] The server sets the interview date and time and sends a reminder email to the user.

[0494] Step 8:

[0495] The user logs in to the interview system through a terminal at the designated interview date and time.

[0496] Step 9:

[0497] The server starts the interview system and the AI ​​interviewer starts asking questions based on a pre-set list of questions.

[0498] Step 10:

[0499] The user answers questions posed by the AI ​​interviewer, and the answers are sent to the server in real time and recorded.

[0500] Step 11:

[0501] The server analyzes the recorded dialogue log and evaluates the user's responses.

[0502] Step 12:

[0503] The server calculates the evaluation results based on the pass / fail criteria and determines whether the application passes or fails.

[0504] Step 13:

[0505] The server stores the pass / fail results in a database.

[0506] Step 14:

[0507] The server creates a notification email containing the pass / fail result and sends it to the user.

[0508] Step 15:

[0509] The user receives a notification email and checks the results.

[0510] The above are the detailed process steps from collecting applicant information to interview evaluation and final notification of results. This flow realizes automation and efficiency of interviews.

[0511] Example 1

[0512] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0513] The existing recruitment interview process requires a great deal of time and effort, and the interviewer's subjectivity can affect the results. There are multiple problems, such as unfair judgments and reduced efficiency due to busy schedules. Furthermore, inconsistent evaluations of applicants make it difficult to select the best candidates. This invention aims to realize a fair and efficient recruitment process.

[0514] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0515] In this invention, the server includes a means for receiving applicant information and storing it in a database, a means for analyzing the recorded dialogue log and evaluating the applicant using natural language processing, and a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result, thereby making it possible to improve the efficiency and fairness of the job interview process.

[0516] "Terminal" means an electronic device used by a user to input and transmit information.

[0517] "Server" means the central system that receives, processes and manages information sent from the terminals.

[0518] A "database" is a storage device within the system that stores and manages data such as applicant information and acceptance criteria.

[0519] The "AI Interviewer" is an artificial intelligence system that interacts with applicants and collects and evaluates their responses based on questions.

[0520] The "interview system" refers to the entire system, including the AI ​​interviewer, that executes and manages the interview process with applicants.

[0521] A "dialogue log" is data that records the content of the conversation between an AI interviewer and an applicant during an interview.

[0522] "Natural language processing" is a technology that allows computers to understand, analyze, and evaluate human language.

[0523] "Pass / fail assessment" is the process of determining whether an applicant passes or fails based on the results of their evaluation.

[0524] "Notification means" refers to the means for informing applicants of the results of their application, such as sending an email.

[0525] In the embodiment for carrying out the invention, this system is a recruitment interview system using AI, which reduces the time and effort required for recruitment interviews on the store side and realizes a fair and efficient recruitment process. This system includes the following elements.

[0526] Collection of applicant information

[0527] The user uses a device (PC, tablet, smartphone, etc.) to open a browser and access the application form. In the application form, they enter information about the applicant, such as their name, available working hours, relevant experience, and self-promotion. The device sends the information entered by the user in JSON format to the server, which then stores the information in a database.

[0528] Setting pass / fail criteria

[0529] The store uses a management terminal to access the management screen and input the pass / fail criteria, such as available working hours, communication skills, relevant experience, etc. The criteria sent from the management screen are received by the server and stored in a database.

[0530] Preparing for and conducting interviews

[0531] The server prepares the interview system based on the applicant's information and pass / fail criteria. At the designated interview date and time, the user accesses the system through their device and begins a dialogue with the AI ​​interviewer. The AI ​​interviewer asks questions from a pre-set list. For example, they may ask, "How many hours per week can you work?" The user responds, and the device sends the answers to the server, which records them in real time.

[0532] Evaluation of interview results

[0533] After the interview, the server analyzes the recorded dialogue log and evaluates the applicant's communication skills using natural language processing. It also checks whether the applicant's answers meet the criteria for acceptance or rejection. For example, it checks whether the applicant's available working hours and related experience meet the criteria. Based on these evaluations, it calculates an overall score and determines whether the applicant is accepted or rejected.

[0534] Notification of results

[0535] After the pass / fail decision is complete, the server notifies the applicant of the result by email. For example, if the applicant passes, an email containing details of the next steps is sent. The user opens the received email, checks the result, and proceeds to the next step.

[0536] For example, consider the following prompt:

[0537] "Please evaluate the applicant's answers based on whether they are available to work 20 hours or more per week, have at least one year of related experience, and have strong communication skills, and calculate an overall score. Based on this score, you will decide whether the applicant has passed or failed, and if successful, you will send an email with details of the next steps."

[0538] The above steps allow stores to evaluate applicants and conduct hiring activities efficiently and fairly, and the system can significantly reduce the time and effort required for interviews.

[0539] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0540] Step 1:

[0541] The user displays the input screen. The user opens a browser on a device (PC, tablet, smartphone, etc.) and accesses the application form. Input: Browser URL Output: Application form screen

[0542] Step 2:

[0543] The user enters applicant information. The user enters the name, available working hours, related experience, self-promotion, etc. into the application form. Input: Applicant information (name, available working hours, related experience, self-promotion) Output: Completed application form

[0544] Step 3:

[0545] The terminal sends applicant information to the server. When the user presses the "Send" button, the terminal sends the entered information to the server in JSON format. Input: Applicant information (JSON format) Output: Data sent to the server

[0546] Step 4:

[0547] The server receives the applicant information and saves it in the database. The server saves the received information in the database and returns a response to the terminal indicating that saving has been completed. Input: Applicant information (JSON format) Output: Saved in the database, save completion response

[0548] Step 5:

[0549] The store displays the management screen. The store accesses the management screen using a management terminal. Input: Browser URL Output: Management screen

[0550] Step 6:

[0551] The store enters the pass / fail criteria. From the management screen, the store enters pass / fail criteria such as available working hours, communication skills, and related experience. Input: Available working hours, communication skills, and related experience criteria Output: Entered pass / fail criteria

[0552] Step 7:

[0553] The terminal sends the pass / fail criteria to the server. When the store presses the "Save" button, the terminal sends the criteria to the server in JSON format. Input: Pass / fail criteria (JSON format) Output: Data sent to the server

[0554] Step 8:

[0555] The server saves the pass / fail criteria in the database. The server saves the received criteria in the database and returns a response to the terminal indicating that saving is complete. Input: Pass / fail criteria (JSON format) Output: Saved in the database, save completion response

[0556] Step 9:

[0557] The server prepares the interview system. Based on the applicant information and pass / fail criteria, the server configures the interview system. Input: Applicant information, pass / fail criteria Output: Interview system is ready

[0558] Step 10:

[0559] The user accesses the system at the specified interview date and time. The user logs in to the interview system through a terminal. Input: Interview login URL Output: Interview screen

[0560] Step 11:

[0561] The AI ​​interviewer asks questions. The AI ​​interviewer on the server selects questions from a pre-set list and presents them to the user. Input: Question list Output: Questions asked by the AI ​​interviewer

[0562] Step 12:

[0563] The user answers the question. The user answers the question by text or voice. Input: User's answer Output: Answer data

[0564] Step 13:

[0565] The server records the answers in real time. The server receives the user's answers in real time and saves them as a dialogue log. Input: Answer data Output: Dialogue log

[0566] Step 14:

[0567] The server analyzes the dialogue log. After the interview, the server analyzes the recorded dialogue log and evaluates the applicant's communication skills and responses. Input: Dialogue log Output: Analysis results

[0568] Step 15:

[0569] The server determines whether the candidate passes or fails based on the evaluation results. The server calculates a total score based on criteria such as communication ability, available working hours, and related experience, and determines whether the candidate passes or fails. Input: Analysis results, pass / fail criteria Output: Pass / fail decision

[0570] Step 16:

[0571] The server will notify the result. The server will send the result of the pass / fail decision to the applicant by email. Input: Pass / fail decision result Output: Notification email

[0572] Step 17:

[0573] The user checks the results. The user opens the received email and checks the pass / fail result and next steps. Input: Notification email Output: Check pass / fail result

[0574] (Application example 1)

[0575] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0576] The traditional hiring process had issues such as time-consuming scheduling between applicants and interviewers, and subjective evaluation of applicants. Additionally, conducting interviews and notifying interviewers of results was time-consuming, making efficient hiring difficult. The goal of this invention is to use AI to solve these problems and realize a fairer and more efficient hiring process.

[0577] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0578] In this invention, the server includes a terminal for inputting applicant information, a means for receiving applicant information sent from the terminal, a database connected to the server and a means for storing pass / fail criteria, and an interview system including an AI interviewer connected to the server, which includes a means for engaging in dialogue with the applicant, a means for recording the dialogue between the AI ​​interviewer and the applicant and evaluating them in real time, a means for making a pass / fail decision based on the evaluation and notifying the applicant of the result via an application, and a means for notifying the applicant of the pass / fail decision and providing information on next steps. This automates the job interview process, enabling efficient and fair recruitment activities. Furthermore, using a generative AI model to evaluate applicants improves the reliability and accuracy of the evaluation.

[0579] "Terminal" means the device used by the applicant to enter information, including smartphones, computers, tablets, etc.

[0580] A "server" is a computer system that receives and processes applicant information sent from a terminal.

[0581] "Database" refers to an information storage system for storing and managing applicant information and acceptance criteria.

[0582] "Pass / fail criteria" are criteria used to evaluate applicants during recruitment, and include working hours, related experience, communication skills, etc.

[0583] The "AI Interviewer" is a virtual interviewer that uses artificial intelligence to converse with applicants and analyze and evaluate the content of the conversation.

[0584] "Recording" means saving the conversation between the AI ​​interviewer and the applicant in digital format.

[0585] "Real-time" means that processing and evaluation occurs immediately during the conversation with the applicant.

[0586] "Evaluation" is the act of judging an applicant's suitability and ability based on the applicant's answers and the content of the dialogue.

[0587] "Pass / fail decision" refers to determining whether an applicant meets the employment criteria based on the evaluation results.

[0588] "Notification of results" refers to the act of informing applicants of the results of the pass / fail decision.

[0589] An "application" is software that runs on a smartphone or other digital device and is used by applicants to receive results or conduct interviews.

[0590] "Procedural information" refers to specific information provided to successful applicants as the next step, including information on required documents and start dates.

[0591] A "generative AI model" is an artificial intelligence model trained to perform natural language processing and data analysis, and is used to analyze and evaluate applicant responses.

[0592] The present invention is a system for automating the job interview process. This system is designed to perform an entire process from inputting applicant information to notifying the interview results. Detailed embodiments of the present invention are described below.

[0593] Hardware and software used

[0594] 1. Device: The device used by the applicant to enter information, including smartphones, computers, tablets, etc. In particular, we will develop cross-platform mobile applications using React Native.

[0595] 2. Server: This is a computer system that receives and processes applicant information sent from the terminal. The server is built using Node.js and Express and performs data processing.

[0596] 3. Database: An information storage system for storing and managing applicant information and acceptance criteria. MySQL is used.

[0597] 4. AI Interviewer: An artificial intelligence that interacts with applicants and uses generative AI models such as GPT-4 to process interview questions and applicant responses.

[0598] 5. Natural language processing system: Natural language processing is implemented using Python to analyze the applicants' responses.

[0599] 6. Application: This is the software that allows applicants to go through the interview process and notify them of the evaluation results. Applicants participate in the interview and receive the results via the application.

[0600] Data processing and calculation

[0601] The server processes and calculates the data as follows:

[0602] 1. The applicant uses a terminal to enter information such as name, available working hours, relevant experience, and self-promotion into the application. This information is then sent to the server via a REST API.

[0603] 2. The server receives this information and stores it in a MySQL database, where each applicant is assigned a unique ID.

[0604] 3. At the designated interview date and time, the applicant accesses the system through the application and begins a dialogue with the AI ​​interviewer. The AI ​​interviewer asks the applicant questions based on a pre-set list of questions and accepts the applicant's answers.

[0605] 4. The AI ​​interviewer (generative AI model) analyzes the applicant's responses in real time and generates a score for evaluation. For example, it checks whether the applicant's available hours and relevant experience meet the set criteria. Communication skills are also evaluated using a natural language processing system.

[0606] 5. The server makes a comprehensive pass / fail decision based on the analyzed dialogue log. The result of this decision is notified to the application used by the applicant. Successful applicants are then informed of the next steps.

[0607] Specific examples

[0608] As a concrete example, let's say an applicant enters the name "Yamada Taro," his available working hours as "20 hours per week," his related experience as "2 years," and his self-promotion as "I'm good at customer service." This information is sent to the server and saved in the database.

[0609] At the time of the interview, the applicant logs in through the application and speaks to the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?" and the applicant replies, "I can work 20 hours per week." This response is recorded in real time and analyzed by GPT-4. Based on the analysis results, the server makes a pass / fail decision and notifies the applicant of the result.

[0610] Prompt Sentence Examples

[0611] Here are some examples of prompts for generative AI models:

[0612] An applicant named "Yamada Taro" exists in the recruitment interview system. His available working hours are "20 hours per week," his related experience is "2 years," and his self-promotional statement is "I'm good at customer service." Based on this information, his communication skills are evaluated according to the following criteria, and a pass / fail decision is made.

[0613] Available working hours: 20 hours or more per week

[0614] Related experience: 1+ years

[0615] Communication skills: High

[0616] Question list:

[0617] 1. "How many hours per week can you work?"

[0618] 2. "What is your previous work experience like?"

[0619] 3. "Please tell us about yourself."

[0620] Based on these questions, analyze Yamada Taro's answers and determine whether he passed or failed.

[0621] Although a detailed description of the present invention has been given, it will be apparent to those skilled in the art that various changes and modifications can be made thereto without departing from the spirit and scope of the invention.

[0622] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0623] Step 1:

[0624] Input: Applicants use a terminal to enter information such as their name, available hours, relevant experience, and personal information into the application.

[0625] How it works: Data entered on the terminal is sent to the server via a REST API through a front-end form.

[0626] Output: The server stores the received applicant information in a database. Each applicant is assigned a unique ID.

[0627] Step 2:

[0628] Input: The server receives the applicant information sent from the terminal.

[0629] How it works: The server uses Node.js and Express to structure the data it receives and store it in a MySQL database.

[0630] Output: The saved data can be accessed from the management screen, and applicant information can be managed centrally.

[0631] Step 3:

[0632] Input: Receive notifications about interview dates and times.

[0633] How it works: The server uses the calendar functionality within the application to send interview date and time reminders to the applicant.

[0634] Output: The applicant receives a notification and accesses the interview system at the specified date and time.

[0635] Step 4:

[0636] Input: The applicant accesses the system at the designated interview date and time and begins a dialogue with the AI ​​interviewer.

[0637] How it works: An AI interviewer (GPT-4) asks applicants questions based on a pre-defined list, and their answers are recorded in real time.

[0638] Output: The applicant's answers are immediately received by the AI ​​interviewer and sent to the server for analysis.

[0639] Step 5:

[0640] Input: The server receives the applicant's response data received from the AI ​​interviewer.

[0641] How it works: The server uses Python for natural language processing to analyze applicants' responses, evaluate their communication skills and other criteria, and calculate an overall score.

[0642] Output: Scores and analysis results are stored in a database.

[0643] Step 6:

[0644] Input: The analysis results and overall score are saved on the server.

[0645] Operation: The server determines whether the applicant passes or fails based on this data, and generates and saves the next procedure information for those who pass.

[0646] Output: A pass / fail result is prepared.

[0647] Step 7:

[0648] Input: The pass / fail result is saved on the server.

[0649] Operation: The server sends the result of the application to the applicant via the notification system. The result details can also be viewed within the applicant's application.

[0650] Output: Applicants receive their results through the application and receive instructions on next steps.

[0651] Example prompt sentence:

[0652] An applicant named "Yamada Taro" exists in the recruitment interview system. His available working hours are "20 hours per week," his related experience is "2 years," and his self-promotional statement is "I'm good at customer service." Based on this information, his communication skills are evaluated according to the following criteria, and a pass / fail decision is made.

[0653] Available working hours: 20 hours or more per week

[0654] Related experience: 1+ years

[0655] Communication skills: High

[0656] Question list:

[0657] 1. "How many hours per week can you work?"

[0658] 2. "What is your previous work experience like?"

[0659] 3. "Please tell us about yourself."

[0660] Based on these questions, analyze Yamada Taro's answers and determine whether he passed or failed.

[0661] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0662] The present invention combines an emotion engine with an AI-based job interview system to further improve the fairness, impartiality, and accuracy of interview evaluations. This system includes a terminal for inputting applicant information, a server for receiving and processing applicant information, a database for storing pass / fail criteria, an AI interviewer who interacts with applicants, a means for recording and analyzing the interaction, and a means for notifying the results. The emotion engine of the present invention also analyzes the user's emotional state and reflects it in the interview questions and evaluations.

[0663] Collection of applicant information

[0664] First, the user (applicant) uses a device (e.g., PC, tablet, smartphone) to enter information such as their name, available working hours, relevant experience, self-promotion, etc. The information sent from the device is received by the server and stored in a database.

[0665] Setting pass / fail criteria

[0666] The store sets the criteria for acceptance through the management screen, including communication skills, available working hours, relevant experience, etc. The criteria entered by the store is received by the server and stored in the database.

[0667] Preparing for and conducting interviews

[0668] The server prepares the interview system based on applicant information and pass / fail criteria. The user accesses the system via their device at the designated interview date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue (questions and answers) is recorded in real time by the server. The AI ​​interviewer asks questions based on a pre-set list of questions, and the user answers them.

[0669] Emotion Engine Operation

[0670] The emotion engine connected to the server analyzes the user's voice tone, facial expressions, and language patterns to determine their emotional state. For example, it detects whether the user's voice sounds tense or relaxed. Based on the user's emotional state, the AI ​​interviewer dynamically adjusts the questions and responses it asks. For example, if the user is nervous, it can add questions to relax them.

[0671] Evaluation of interview results

[0672] The server evaluates the user's responses based on the recorded dialogue log and the results of emotion analysis by the emotion engine. It checks whether the available working hours meet the criteria and whether the relevant experience is appropriate. It evaluates communication skills using natural language processing, and also takes into account the results of emotion analysis by the emotion engine. This calculates an overall score and determines whether the candidate passes or fails.

[0673] Notification of results

[0674] After the pass / fail decision is complete, the server notifies the applicant of the result, for example by sending an email to the user with detailed instructions on what to do if they pass. The user receives their result and confirms the next steps.

[0675] As a specific example, the following flow can be considered.

[0676] 1. Entering applicant information: The user uses a terminal to enter the name "Yamada Taro," available working hours "20 hours per week," related experience "2 years," and self-promotion "I'm good at customer service."

[0677] 2. Receiving and storing: The application information sent from the device is received by the server and stored in a database.

[0678] 3. Setting standards: The store sets the available working hours as "20 hours or more per week," related experience as "1 year or more," and communication skills as "high" on the management screen.

[0679] 4. Conducting the interview: The user logs in to the system on the date and time of the interview and interacts with the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?", and the user replies, "I can work 20 hours per week."

[0680] 5. Emotion analysis: The server's emotion engine detects nervousness from the user's tone of voice and facial expressions, and changes the AI ​​interviewer's questions to make them more relaxed.

[0681] 6. Evaluation of results: The server evaluates the dialogue log and the emotion analysis results, calculates an overall score, and determines whether the user passes or fails.

[0682] 7. Notification of result: The server creates a success notification email and sends it to the user. The user receives the success notification and confirms the next steps.

[0683] As described above, the present invention further promotes automation and efficiency of interviews, and enables more precise evaluation and response that takes into account the user's emotional state. This allows stores to conduct efficient and fair recruitment activities.

[0684] The processing flow will be explained below.

[0685] Step 1:

[0686] The store logs in to the management screen and enters the pass / fail criteria (e.g., communication skills, available working hours, related experience, etc.).

[0687] Step 2:

[0688] The terminal transmits the entered pass / fail criteria to the server.

[0689] Step 3:

[0690] The server saves the received pass / fail criteria in the database. After saving is complete, a confirmation message is displayed to the store.

[0691] Step 4:

[0692] The user enters information such as name, available working hours, relevant experience, and self-promotion into the application form.

[0693] Step 5:

[0694] The terminal transmits the input application information to the server.

[0695] Step 6:

[0696] The server receives the application information and saves it in the database. After saving is complete, a confirmation message is displayed to the user confirming the application.

[0697] Step 7:

[0698] The server sets the interview date and time and sends a reminder email to the user.

[0699] Step 8:

[0700] The user logs in to the interview system through a terminal at the designated interview date and time.

[0701] Step 9:

[0702] The server starts the interview system and the AI ​​interviewer starts asking questions based on a pre-set list of questions.

[0703] Step 10:

[0704] The user answers questions posed by the AI ​​interviewer, and the answers are sent to the server in real time and recorded.

[0705] Step 11:

[0706] An emotion engine connected to the server analyzes the user's voice tone, facial expressions, and language patterns. For example, if a user answers, "I'm a little nervous, but I want to try this job," the emotion engine will detect nervousness from the user's voice tone and facial expressions.

[0707] Step 12:

[0708] The emotion engine determines the user's emotional state based on the analysis results, generating a result such as "I'm nervous."

[0709] Step 13:

[0710] The server receives the results of the sentiment analysis and dynamically adjusts the questions asked by the AI ​​interviewer, for example, changing the question to, "Let's relax a bit. What has been the most rewarding experience you've had so far?"

[0711] Step 14:

[0712] The user answers a new question, and this answer is also sent in real time to the server and recorded.

[0713] Step 15:

[0714] The server evaluates the user's responses based on the dialogue log and the results of emotion analysis by the emotion engine, for example, checking whether the available working hours meet the criteria and whether the relevant experience is appropriate.

[0715] Step 16:

[0716] The server uses natural language processing to evaluate communication skills, and then incorporates the results of the emotion engine analysis into the evaluation to calculate an overall score.

[0717] Step 17:

[0718] The server determines whether the application passes or fails based on the pass / fail criteria.

[0719] Step 18:

[0720] The server stores the pass / fail results in a database.

[0721] Step 19:

[0722] The server creates a notification email containing the pass / fail result and sends it to the user.

[0723] Step 20:

[0724] The user receives a notification email confirming the result, for example, "Congratulations, you passed!" along with instructions on what to do next.

[0725] The above are the detailed processing steps that combine the emotion engine to collect applicant information, evaluate the interview, and notify the final result. This flow realizes automation and efficiency of interviews, and enables more precise evaluation and response that takes into account the user's emotional state.

[0726] Example 2

[0727] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0728] Conventional recruitment interview systems are often influenced by the interviewer's subjectivity and emotions, which can compromise fairness and impartiality. Furthermore, there is a lack of means to properly grasp the applicant's emotional state, which can lead to a decline in the quality of responses and questions during the interview process. Furthermore, there is no system in place to improve the accuracy of evaluations, resulting in a lack of reliability in hiring decisions. There is a need to solve these problems and conduct fair and efficient recruitment activities.

[0729] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a terminal for inputting applicant information, means for receiving applicant information transmitted from the terminal, a database connected to the server and means for storing pass / fail criteria, an AI system connected to the server and means for engaging in a dialogue with the applicant, and an emotion analysis device connected to the server and means for analyzing the user's emotional state, means for recording the dialogue and making an evaluation, and means for making a pass / fail decision based on the evaluation and notifying the applicant of the result. This enables fair and reliable hiring decisions that take into account the applicant's emotional state.

[0730] A "terminal for inputting applicant information" is a device that allows a user to input their own personal information and job-related data, and includes a PC, tablet, smartphone, etc.

[0731] A "server" is a computer system that receives and sends data over a network and manages and controls the operation of databases and various applications.

[0732] "Database" means a system for efficiently managing and accessing recorded information, including the means for storing applicant information and acceptance criteria.

[0733] "Means for storing acceptance criteria" refers to a data storage system for recording and managing the criteria for employment, including criteria such as available working hours and relevant experience.

[0734] An "AI system" is a system that uses artificial intelligence technology to automatically perform specific tasks and analyses, and includes a means for interacting with applicants.

[0735] An "emotion analysis device" is a device that analyzes a user's voice tone, facial expressions, language patterns, etc. to determine their emotional state.

[0736] "Means for recording and evaluating the dialogue" refers to software or hardware for recording the dialogue between the applicant and the AI ​​system and evaluating that content.

[0737] The "means for determining whether an applicant passes or fails and notifying the applicant of the result" is a system for determining whether an applicant passes or fails based on the content of the dialogue and the evaluation results, and notifying the applicant of the result.

[0738] This invention is a system that improves fairness and evaluation accuracy by combining emotion analysis technology with an AI-based recruitment interview system. This system consists of the following main hardware and software:

[0739] System Configuration

[0740] 1. Terminal: A device on which users enter their application information and interact with the AI ​​interviewer. Terminals include PCs, tablets, smartphones, etc.

[0741] 2. Server: A central computer system that receives, processes, and stores application information, and also integrates and manages AI systems and emotion analysis devices.

[0742] 3. Database: A data storage system for storing applicant information and acceptance criteria.

[0743] 4. AI System: An artificial intelligence-based system for interacting with applicants.

[0744] 5. Emotion analysis device: A device that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[0745] Collection of applicant information

[0746] The user uses the terminal to input information such as name, available working hours, related experience, and self-promotion. For example, the user might input "Yamada Taro," "Available 20 hours per week," "2 years of related experience," and "I'm good at customer service." The input information is sent from the terminal to the server and stored in a database.

[0747] Setting pass / fail criteria

[0748] The store sets the pass / fail criteria for the available working hours, related experience, communication skills, etc. through the management screen. For example, they can set "20 hours or more per week," "1 year or more of related experience," and "high communication skills." These criteria are sent from the terminal to the server and stored in a database.

[0749] Preparing for and conducting interviews

[0750] The server prepares the interview system based on the applicant information and pass / fail criteria. The user accesses the system from their device at the specified date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue is recorded on the server in real time. For example, the AI ​​interviewer asks, "How many hours per week can you work?" and the user replies, "I can work 20 hours per week."

[0751] Emotion analysis

[0752] The emotion analyzer analyzes the user's tone of voice and facial expressions to determine their emotional state. For example, if the AI ​​interviewer detects that the user is nervous, it will add questions to help them relax. It dynamically changes the questions to include, "Is there anything you can do to relax?"

[0753] Evaluation of interview results

[0754] The server comprehensively evaluates the user's responses based on the recorded dialogue log and the results of sentiment analysis. It checks whether the user's available working hours and related experience meet the criteria, and evaluates their communication skills using natural language processing. For example, a user with strong communication skills will respond calmly and clearly.

[0755] Notification of results

[0756] After the pass / fail decision is complete, the server notifies the applicant of the result. For example, if the applicant passes, an email is sent to the user saying, "Congratulations! Here are the detailed steps to proceed to the next step." The user receives the notification and confirms the next steps.

[0757] This system automates the hiring process and enables precise evaluations that take into account the emotional state of applicants, allowing stores to conduct hiring activities efficiently and fairly.

[0758] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0759] Step 1:

[0760] The user uses a terminal to input application information. The input information includes name, available working hours, related experience, self-promotion, etc. For example, the user might input "Yamada Taro," "Available 20 hours per week," "2 years of related experience," and "I'm good at customer service." The input data is temporarily saved on the terminal.

[0761] Input: Application information entered by the user into the device

[0762] Output: Application information temporarily saved on the device

[0763] Step 2:

[0764] The terminal sends the application information to the server. The terminal formats the input data and sends it to the server in the form of an HTTP request or similar. The server receives this request.

[0765] Input: Application information temporarily saved on your device

[0766] Output: Application information sent to the server

[0767] Step 3:

[0768] The server stores the received application information in a database. During this process, the server uses a database management system to check the integrity of the data and store the applicant information securely.

[0769] Input: Application information sent to the server

[0770] Output: Application information stored in a database

[0771] Step 4:

[0772] The store sets the pass / fail criteria through the management screen. For example, they can input criteria such as available working hours of "20 hours or more per week," related experience of "1 year or more," and communication skills of "high." The input criteria are sent from the terminal to the server.

[0773] Input: Pass / fail criteria entered by the store into the terminal

[0774] Output: Pass / fail criteria sent to the server

[0775] Step 5:

[0776] The server stores the received pass / fail criteria in a database, at which point the server checks the integrity of the data and adds the criteria to the database.

[0777] Input: Pass / fail criteria sent to the server

[0778] Output: Pass / fail criteria stored in the database

[0779] Step 6:

[0780] As the interview date and time approaches, the server configures the interview system based on the applicant's information and the criteria for acceptance or rejection, including the list of questions for the AI ​​interviewer and the interview schedule.

[0781] Input: Application information and acceptance criteria stored in the database

[0782] Output: Interview system settings

[0783] Step 7:

[0784] The user accesses the system through a terminal at the specified date and time and begins a conversation with the AI ​​interviewer. The terminal sends a connection request to the server, and the server launches the AI ​​interviewer.

[0785] Input: User access request

[0786] Output: Launched AI interviewer

[0787] Step 8:

[0788] The AI ​​interviewer asks questions to the applicant, and the user answers. This dialogue is sent to the server in real time. For example, the AI ​​interviewer asks, "How many hours can you work per week?" and the user answers, "I can work 20 hours per week."

[0789] Input: User's answer

[0790] Output: Real-time conversation logs recorded on the server

[0791] Step 9:

[0792] The emotion analyzer analyzes the user's tone of voice and facial expressions to determine their emotional state. For example, if the server detects that the user is nervous, it will dynamically change the AI ​​interviewer's questions to make them more relaxed.

[0793] Input: User's voice tone and facial expression data

[0794] Output: Emotional state analysis result

[0795] Step 10:

[0796] The server evaluates the user's responses based on the recorded dialogue logs and sentiment analysis results. This evaluation involves determining whether the user's available working hours and relevant experience meet the criteria, and using natural language processing to evaluate their communication skills.

[0797] Input: Dialogue logs and sentiment analysis results

[0798] Output: Evaluation results and overall score

[0799] Step 11:

[0800] After the pass / fail decision is complete, the server notifies the applicant of the result. For example, if the applicant passes, an email is sent to the user saying, "Congratulations! Here are the detailed steps to proceed to the next step." The user can then check the result on their device.

[0801] Input: Evaluation results and overall score

[0802] Output: Result email sent to applicant

[0803] (Application example 2)

[0804] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0805] Conventional recruitment interview systems did not take into account the emotional state of applicants during evaluation, resulting in problems with the fairness and impartiality of the interview. Furthermore, they were unable to analyze the emotional state of passengers in real time and optimize the in-car environment accordingly. This resulted in passengers' riding experiences not always being satisfactory. Therefore, the challenge is to provide a recruitment interview system that enables emotional analysis of applicants and provides fair and precise evaluations, as well as a system that provides a comfortable riding experience that reflects the emotional state of passengers.

[0806] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0807] In this invention, the server includes a terminal for inputting applicant information, a server that receives the applicant information transmitted from the terminal, a database connected to the server and having means for storing pass / fail criteria, and an interview system including an AI interviewer connected to the server and having means for engaging in a dialogue with the applicant, means for recording the dialogue and making an evaluation, means for analyzing the emotional state of passengers and dynamically adjusting the in-car environment, and means for making a pass / fail decision based on the evaluation and notifying the applicant of the result. This enables fair and precise evaluation that takes into account the emotional state of the applicant, and also enables the in-car environment to be optimized based on the emotional state of passengers, resulting in a more satisfying riding experience.

[0808] A "terminal" is a device through which applicants or customers enter information.

[0809] "Server" means a central computer group that receives and processes information sent from the Terminals.

[0810] "Database" means a storage device for storing pass / fail criteria and other relevant information.

[0811] An "AI interviewer" is software based on artificial intelligence that is used to interact with applicants.

[0812] The "interview system" is a set of mechanisms that includes an AI interviewer to interact with applicants, record the interaction, and evaluate it.

[0813] "Means for conducting evaluation" means a method or device for evaluating the abilities and qualifications of an applicant based on the dialogue records or other criteria.

[0814] The "means for determining whether an applicant passes or fails and notifying the applicant of the result" refers to a method or device for determining whether an applicant passes or fails based on the evaluation and notifying the applicant of the result.

[0815] The "means for analyzing emotional state" refers to a method or apparatus that analyzes voice tone and facial expressions to determine the user's emotions.

[0816] The "means for dynamically adjusting the in-car environment" refers to a method or device that automatically changes the background music, temperature, lighting, seat angle, etc. in the car based on the user's emotional state.

[0817] "Riding customers" are people riding in the autonomous vehicle.

[0818] To implement this invention, a terminal for inputting applicant information is first prepared. The terminal can be a smartphone, tablet, or PC. The applicant uses the terminal to input information such as their name, available working hours, relevant experience, and self-promotion. The input information is sent to a server, which receives it and stores it in a database.

[0819] Next, the server receives the pass / fail criteria from the administrator and stores them in the database. The administrator sets the criteria such as available working hours, relevant experience, communication ability, etc. The pass / fail criteria are important indicators used to evaluate the job interview.

[0820] The server prepares an AI interviewer based on applicant information and pass / fail criteria. The AI ​​interviewer uses natural language processing technology to converse with the applicant. During the conversation, the server activates an emotion engine that analyzes the applicant's tone of voice and facial expressions to understand the applicant's emotional state. The emotion engine is built using software such as TensorFlow and OpenCV. For example, it can add questions to relax nervous applicants.

[0821] After the conversation is over, the server evaluates the applicant based on the content of the conversation and the results of sentiment analysis. Using natural language processing technology, the server evaluates the applicant's communication skills and calculates an overall score based on criteria such as available working hours and relevant experience. Based on this, the server determines whether the applicant is successful and notifies the applicant of the result.

[0822] The system also has the functionality to be used in autonomous vehicles. It can analyze the emotional state of passengers and dynamically adjust the in-car environment. Using the smartphone's camera and microphone, it analyzes the passenger's facial expressions and tone of voice and automatically adjusts the in-car background music, temperature, lighting, seat angle, and more, providing the optimal experience for passengers while they are in the vehicle.

[0823] Examples:

[0824] After a user gets into an autonomous vehicle, they launch the "Emotion Reader" app. The app uses the smartphone's camera and microphone to analyze the user's emotional state in real time. For example, if the app detects that the user's facial expression indicates "surprise" or "tension," it automatically adjusts the in-car background music to relaxing music. Furthermore, after a long ride, the app reports any changes in the user's emotions (from tension to relaxation).

[0825] Hardware and software used:

[0826] Smartphone: A device equipped with a camera and microphone to analyze the emotional state of passengers.

[0827] Python: The primary programming language used.

[0828] OpenCV: Used for real-time analysis of camera footage.

[0829] TensorFlow: Used to build the sentiment analysis model.

[0830] Firebase: Used for database and real-time updates.

[0831] Flask: A microframework for providing a sentiment analysis API on the backend.

[0832] Example prompt sentence:

[0833] "Tell me about a system in which a smartphone app inside a self-driving vehicle uses cameras and microphones to analyze passenger emotions in real time and optimize the ride experience (background music, temperature, lighting)."

[0834] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0835] Step 1:

[0836] Input: The applicant enters their name, available working hours, relevant experience, and self-promotional information into the terminal.

[0837] Output: The entered information is sent to the server.

[0838] Specific operation: The user enters the required information into the form on the device and presses the submit button. The device converts the entered information into JSON format and sends it to the server. The server saves the received information in a database in real time.

[0839] Step 2:

[0840] Input: The administrator inputs the pass / fail criteria from the terminal.

[0841] Output: The pass / fail criteria is sent to the server and stored in a database.

[0842] Specific operation: The administrator sets criteria such as available working hours, relevant experience, and communication skills on the management screen and sends them to the server, which receives them and stores them in the database.

[0843] Step 3:

[0844] Input: The server obtains applicant information and acceptance criteria.

[0845] Output: The AI ​​interviewer is prepared.

[0846] Specific operation: The server retrieves applicant information and pass / fail criteria from the database, and prepares an AI interviewer based on that information. The AI ​​interviewer loads a pre-set list of questions.

[0847] Step 4:

[0848] Input: The applicant uses the terminal to access the interview system.

[0849] Output: A conversation between the AI ​​interviewer and the applicant begins.

[0850] Specific operation: The applicant logs in to the system from their terminal at the designated interview date and time. The server authenticates the applicant and starts a dialogue session with the AI ​​interviewer. The dialogue is recorded in real time.

[0851] Step 5:

[0852] Input: AI interviewer analyzes applicant's tone of voice and facial expressions.

[0853] Output: Sentiment analysis results are obtained.

[0854] Specific operation: The server analyzes the camera footage using OpenCV to recognize the applicant's facial expressions, and analyzes the voice data using a TensorFlow model to analyze the voice tone. The analysis results are saved on the server.

[0855] Step 6:

[0856] Input: Dialogue records and sentiment analysis results are collected on the server.

[0857] Output: The applicant's overall rating is calculated.

[0858] How it works: The server uses natural language processing technology to evaluate the applicant's communication skills based on the dialogue log and sentiment analysis results, and calculates an overall score based on criteria such as available working hours and related experience.

[0859] Step 7:

[0860] Input: The server will determine whether the candidate passes or fails based on the overall evaluation.

[0861] Output: The applicant is notified of the results.

[0862] Specific operation: The server determines whether the applicant has passed or failed based on the overall evaluation score, and notifies the applicant of the result by email. The applicant receives the result and confirms the next steps.

[0863] Step 8:

[0864] Input: A customer enters an autonomous vehicle and launches a smartphone app.

[0865] Output: The in-car environment is dynamically adjusted based on the customer's emotional state.

[0866] How it works: The user launches the "Emotion Reader" smartphone app while in the car. The smartphone's camera and microphone are used to analyze the customer's facial expressions and tone of voice in real time. Based on the analysis results, the car's background music, temperature, lighting, seat angle, and other settings are automatically adjusted.

[0867] Example prompt sentence:

[0868] "Tell me about a system in which a smartphone app inside a self-driving vehicle uses cameras and microphones to analyze passenger emotions in real time and optimize the ride experience (background music, temperature, lighting)."

[0869] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0870] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0871] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0872] [Third embodiment]

[0873] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0874] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0875] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0876] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0877] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0878] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0879] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0880] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0881] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0882] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0883] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0884] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0885] This invention is a recruitment interview system that uses AI, which reduces the time and effort required for job interviews at stores and realizes a fair and efficient recruitment process. The system includes a terminal for inputting applicant information, a server for receiving and processing applicant information, a database for storing pass / fail criteria, an AI interviewer who interacts with applicants, a means for recording and analyzing the interaction, and a means for notifying the results.

[0886] Collection of applicant information

[0887] First, the user (applicant) uses a device (e.g., PC, tablet, smartphone) to enter the necessary information into the application form. This information includes name, available working hours, relevant experience, self-promotion, etc. The information sent from the device is received by the server and stored in a database.

[0888] Setting pass / fail criteria

[0889] The store sets various criteria for acceptance through the management screen, such as communication skills, available working hours, relevant experience, etc. The criteria entered by the store are received by the server and stored in the database.

[0890] Preparing for and conducting interviews

[0891] The server prepares the interview system based on applicant information and pass / fail criteria. The user accesses the system via their device at the designated interview date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue (questions and answers) is recorded in real time by the server. The AI ​​interviewer asks questions based on a pre-set list of questions and accepts the user's answers.

[0892] Evaluation of interview results

[0893] The server analyzes the recorded dialogue logs and evaluates the user's responses. For example, it checks whether the available working hours meet the criteria and whether the relevant experience is appropriate. It also evaluates the user's communication skills using natural language processing. Based on these evaluations, the server calculates an overall score and determines whether the user is accepted or rejected.

[0894] Notification of results

[0895] After the pass / fail decision is complete, the server notifies the applicant of the result, for example by sending an email to the applicant with details of the next steps if they pass. The user receives their result and can confirm the next steps.

[0896] As a specific example, the following flow can be considered.

[0897] 1. Entering applicant information: The user uses a terminal to enter the name "Yamada Taro," available working hours "20 hours per week," related experience "2 years," and self-promotion "I'm good at customer service."

[0898] 2. Receiving and storing: The application information sent from the device is received by the server and stored in a database.

[0899] 3. Setting standards: The store sets the available working hours as "20 hours or more per week," related experience as "1 year or more," and communication skills as "high" on the management screen.

[0900] 4. Conducting the interview: The user logs in to the system on the date and time of the interview and interacts with the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?", and the user replies, "I can work 20 hours per week."

[0901] 5. Evaluation of results: The server analyzes the dialogue log, compares the user's available working hours and related experience with the standard, and evaluates their communication skills using natural language processing.

[0902] 6. Pass / fail determination and notification: The server calculates the applicant's overall score, determines whether they passed or failed, and notifies the user of the result by email.

[0903] The above steps allow stores to evaluate applicants and conduct hiring activities efficiently and fairly, and the system can significantly reduce the time and effort required for interviews.

[0904] The processing flow will be explained below.

[0905] Step 1:

[0906] The store logs in to the management screen and enters the pass / fail criteria (e.g., communication skills, available working hours, related experience, etc.).

[0907] Step 2:

[0908] The terminal transmits the entered pass / fail criteria to the server.

[0909] Step 3:

[0910] The server saves the received pass / fail criteria in the database. After saving is complete, a confirmation message is displayed to the store.

[0911] Step 4:

[0912] The user enters information such as name, available working hours, relevant experience, and self-promotion into the application form.

[0913] Step 5:

[0914] The terminal transmits the input application information to the server.

[0915] Step 6:

[0916] The server receives the application information and saves it in the database. After saving is complete, a confirmation message is displayed to the user confirming the application.

[0917] Step 7:

[0918] The server sets the interview date and time and sends a reminder email to the user.

[0919] Step 8:

[0920] The user logs in to the interview system through a terminal at the designated interview date and time.

[0921] Step 9:

[0922] The server starts the interview system and the AI ​​interviewer starts asking questions based on a pre-set list of questions.

[0923] Step 10:

[0924] The user answers questions posed by the AI ​​interviewer, and the answers are sent to the server in real time and recorded.

[0925] Step 11:

[0926] The server analyzes the recorded dialogue log and evaluates the user's responses.

[0927] Step 12:

[0928] The server calculates the evaluation results based on the pass / fail criteria and determines whether the application passes or fails.

[0929] Step 13:

[0930] The server stores the pass / fail results in a database.

[0931] Step 14:

[0932] The server creates a notification email containing the pass / fail result and sends it to the user.

[0933] Step 15:

[0934] The user receives a notification email and checks the results.

[0935] The above are the detailed process steps from collecting applicant information to interview evaluation and final notification of results. This flow realizes automation and efficiency of interviews.

[0936] Example 1

[0937] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0938] The existing recruitment interview process requires a great deal of time and effort, and the interviewer's subjectivity can affect the results. There are multiple problems, such as unfair judgments and reduced efficiency due to busy schedules. Furthermore, inconsistent evaluations of applicants make it difficult to select the best candidates. This invention aims to realize a fair and efficient recruitment process.

[0939] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0940] In this invention, the server includes a means for receiving applicant information and storing it in a database, a means for analyzing the recorded dialogue log and evaluating the applicant using natural language processing, and a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result, thereby making it possible to improve the efficiency and fairness of the job interview process.

[0941] "Terminal" means an electronic device used by a user to input and transmit information.

[0942] "Server" means the central system that receives, processes and manages information sent from the terminals.

[0943] A "database" is a storage device within the system that stores and manages data such as applicant information and acceptance criteria.

[0944] The "AI Interviewer" is an artificial intelligence system that interacts with applicants and collects and evaluates their responses based on questions.

[0945] The "interview system" refers to the entire system, including the AI ​​interviewer, that executes and manages the interview process with applicants.

[0946] A "dialogue log" is data that records the content of the conversation between an AI interviewer and an applicant during an interview.

[0947] "Natural language processing" is a technology that allows computers to understand, analyze, and evaluate human language.

[0948] "Pass / fail assessment" is the process of determining whether an applicant passes or fails based on the results of their evaluation.

[0949] "Notification means" refers to the means for informing applicants of the results of their application, such as sending an email.

[0950] In the embodiment for carrying out the invention, this system is a recruitment interview system using AI, which reduces the time and effort required for recruitment interviews on the store side and realizes a fair and efficient recruitment process. This system includes the following elements.

[0951] Collection of applicant information

[0952] The user uses a device (PC, tablet, smartphone, etc.) to open a browser and access the application form. In the application form, they enter information about the applicant, such as their name, available working hours, relevant experience, and self-promotion. The device sends the information entered by the user in JSON format to the server, which then stores the information in a database.

[0953] Setting pass / fail criteria

[0954] The store uses a management terminal to access the management screen and input the pass / fail criteria, such as available working hours, communication skills, relevant experience, etc. The criteria sent from the management screen are received by the server and stored in a database.

[0955] Preparing for and conducting interviews

[0956] The server prepares the interview system based on the applicant's information and pass / fail criteria. At the designated interview date and time, the user accesses the system through their device and begins a dialogue with the AI ​​interviewer. The AI ​​interviewer asks questions from a pre-set list. For example, they may ask, "How many hours per week can you work?" The user responds, and the device sends the answers to the server, which records them in real time.

[0957] Evaluation of interview results

[0958] After the interview, the server analyzes the recorded dialogue log and evaluates the applicant's communication skills using natural language processing. It also checks whether the applicant's answers meet the criteria for acceptance or rejection. For example, it checks whether the applicant's available working hours and related experience meet the criteria. Based on these evaluations, it calculates an overall score and determines whether the applicant is accepted or rejected.

[0959] Notification of results

[0960] After the pass / fail decision is complete, the server notifies the applicant of the result by email. For example, if the applicant passes, an email containing details of the next steps is sent. The user opens the received email, checks the result, and proceeds to the next step.

[0961] For example, consider the following prompt:

[0962] "Please evaluate the applicant's answers based on whether they are available to work 20 hours or more per week, have at least one year of related experience, and have strong communication skills, and calculate an overall score. Based on this score, you will decide whether the applicant has passed or failed, and if successful, you will send an email with details of the next steps."

[0963] The above steps allow stores to evaluate applicants and conduct hiring activities efficiently and fairly, and the system can significantly reduce the time and effort required for interviews.

[0964] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0965] Step 1:

[0966] The user displays the input screen. The user opens a browser on a device (PC, tablet, smartphone, etc.) and accesses the application form. Input: Browser URL Output: Application form screen

[0967] Step 2:

[0968] The user enters applicant information. The user enters the name, available working hours, related experience, self-promotion, etc. into the application form. Input: Applicant information (name, available working hours, related experience, self-promotion) Output: Completed application form

[0969] Step 3:

[0970] The terminal sends applicant information to the server. When the user presses the "Send" button, the terminal sends the entered information to the server in JSON format. Input: Applicant information (JSON format) Output: Data sent to the server

[0971] Step 4:

[0972] The server receives the applicant information and saves it in the database. The server saves the received information in the database and returns a response to the terminal indicating that saving has been completed. Input: Applicant information (JSON format) Output: Saved in the database, save completion response

[0973] Step 5:

[0974] The store displays the management screen. The store accesses the management screen using a management terminal. Input: Browser URL Output: Management screen

[0975] Step 6:

[0976] The store enters the pass / fail criteria. From the management screen, the store enters pass / fail criteria such as available working hours, communication skills, and related experience. Input: Available working hours, communication skills, and related experience criteria Output: Entered pass / fail criteria

[0977] Step 7:

[0978] The terminal sends the pass / fail criteria to the server. When the store presses the "Save" button, the terminal sends the criteria to the server in JSON format. Input: Pass / fail criteria (JSON format) Output: Data sent to the server

[0979] Step 8:

[0980] The server saves the pass / fail criteria in the database. The server saves the received criteria in the database and returns a response to the terminal indicating that saving is complete. Input: Pass / fail criteria (JSON format) Output: Saved in the database, save completion response

[0981] Step 9:

[0982] The server prepares the interview system. Based on the applicant information and pass / fail criteria, the server configures the interview system. Input: Applicant information, pass / fail criteria Output: Interview system is ready

[0983] Step 10:

[0984] The user accesses the system at the specified interview date and time. The user logs in to the interview system through a terminal. Input: Interview login URL Output: Interview screen

[0985] Step 11:

[0986] The AI ​​interviewer asks questions. The AI ​​interviewer on the server selects questions from a pre-set list and presents them to the user. Input: Question list Output: Questions asked by the AI ​​interviewer

[0987] Step 12:

[0988] The user answers the question. The user answers the question by text or voice. Input: User's answer Output: Answer data

[0989] Step 13:

[0990] The server records the answers in real time. The server receives the user's answers in real time and saves them as a dialogue log. Input: Answer data Output: Dialogue log

[0991] Step 14:

[0992] The server analyzes the dialogue log. After the interview, the server analyzes the recorded dialogue log and evaluates the applicant's communication skills and responses. Input: Dialogue log Output: Analysis results

[0993] Step 15:

[0994] The server determines whether the candidate passes or fails based on the evaluation results. The server calculates a total score based on criteria such as communication ability, available working hours, and related experience, and determines whether the candidate passes or fails. Input: Analysis results, pass / fail criteria Output: Pass / fail decision

[0995] Step 16:

[0996] The server will notify the result. The server will send the result of the pass / fail decision to the applicant by email. Input: Pass / fail decision result Output: Notification email

[0997] Step 17:

[0998] The user checks the results. The user opens the received email and checks the pass / fail result and next steps. Input: Notification email Output: Check pass / fail result

[0999] (Application example 1)

[1000] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1001] The traditional hiring process had issues such as time-consuming scheduling between applicants and interviewers, and subjective evaluation of applicants. Additionally, conducting interviews and notifying interviewers of results was time-consuming, making efficient hiring difficult. The present invention aims to solve these problems by utilizing AI, realizing a fairer and more efficient hiring process.

[1002] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1003] In this invention, the server includes a terminal for inputting applicant information, a means for receiving applicant information sent from the terminal, a database connected to the server and a means for storing pass / fail criteria, and an interview system including an AI interviewer connected to the server, which includes a means for engaging in dialogue with the applicant, a means for recording the dialogue between the AI ​​interviewer and the applicant and evaluating them in real time, a means for making a pass / fail decision based on the evaluation and notifying the applicant of the result via an application, and a means for notifying the applicant of the pass / fail decision and providing information on next steps. This automates the job interview process, enabling efficient and fair recruitment activities. Furthermore, using a generative AI model to evaluate applicants improves the reliability and accuracy of the evaluation.

[1004] "Terminal" means the device used by the applicant to enter information, including smartphones, computers, tablets, etc.

[1005] A "server" is a computer system that receives and processes applicant information sent from a terminal.

[1006] "Database" refers to an information storage system for storing and managing applicant information and acceptance criteria.

[1007] "Pass / fail criteria" are criteria used to evaluate applicants during recruitment, and include working hours, related experience, communication skills, etc.

[1008] The "AI Interviewer" is a virtual interviewer that uses artificial intelligence to converse with applicants and analyze and evaluate the content of the conversation.

[1009] "Recording" means saving the conversation between the AI ​​interviewer and the applicant in digital format.

[1010] "Real-time" means that processing and evaluation occurs immediately during the conversation with the applicant.

[1011] "Evaluation" is the act of judging an applicant's suitability and ability based on the applicant's answers and the content of the dialogue.

[1012] "Pass / fail decision" refers to determining whether an applicant meets the employment criteria based on the evaluation results.

[1013] "Notification of results" refers to the act of informing applicants of the results of the pass / fail decision.

[1014] An "application" is software that runs on a smartphone or other digital device and is used by applicants to receive results or conduct interviews.

[1015] "Procedural information" refers to specific information provided to successful applicants as the next step, including information on required documents and start dates.

[1016] A "generative AI model" is an artificial intelligence model trained to perform natural language processing and data analysis, and is used to analyze and evaluate applicant responses.

[1017] The present invention is a system for automating the job interview process. This system is designed to perform an entire process from inputting applicant information to notifying the interview results. Detailed embodiments of the present invention are described below.

[1018] Hardware and software used

[1019] 1. Device: The device used by the applicant to enter information, including smartphones, computers, tablets, etc. In particular, we will develop cross-platform mobile applications using React Native.

[1020] 2. Server: This is a computer system that receives and processes applicant information sent from the terminal. The server is built using Node.js and Express and performs data processing.

[1021] 3. Database: An information storage system for storing and managing applicant information and acceptance criteria. MySQL is used.

[1022] 4. AI Interviewer: An artificial intelligence that interacts with applicants and uses generative AI models such as GPT-4 to process interview questions and applicant responses.

[1023] 5. Natural language processing system: Natural language processing is implemented using Python to analyze the applicants' responses.

[1024] 6. Application: This is the software that allows applicants to go through the interview process and notify them of the evaluation results. Applicants participate in the interview and receive the results via the application.

[1025] Data processing and calculation

[1026] The server processes and calculates the data as follows:

[1027] 1. The applicant uses a terminal to enter information such as name, available working hours, relevant experience, and self-promotion into the application. This information is then sent to the server via a REST API.

[1028] 2. The server receives this information and stores it in a MySQL database, where each applicant is assigned a unique ID.

[1029] 3. At the designated interview date and time, the applicant accesses the system through the application and begins a dialogue with the AI ​​interviewer. The AI ​​interviewer asks the applicant questions based on a pre-set list of questions and accepts the applicant's answers.

[1030] 4. The AI ​​interviewer (generative AI model) analyzes the applicant's responses in real time and generates a score for evaluation. For example, it checks whether the applicant's available hours and relevant experience meet the set criteria. Communication skills are also evaluated using a natural language processing system.

[1031] 5. The server makes a comprehensive pass / fail decision based on the analyzed dialogue log. The result of this decision is notified to the application used by the applicant. Successful applicants are then informed of the next steps.

[1032] Specific examples

[1033] As a concrete example, let's say an applicant enters the name "Yamada Taro," his available working hours as "20 hours per week," his related experience as "2 years," and his self-promotion as "I'm good at customer service." This information is sent to the server and saved in the database.

[1034] At the time of the interview, the applicant logs in through the application and speaks to the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?" and the applicant replies, "I can work 20 hours per week." This response is recorded in real time and analyzed by GPT-4. Based on the analysis results, the server makes a pass / fail decision and notifies the applicant of the result.

[1035] Prompt Sentence Examples

[1036] Here are some examples of prompts for generative AI models:

[1037] An applicant named "Yamada Taro" exists in the recruitment interview system. His available working hours are "20 hours per week," his related experience is "2 years," and his self-promotional statement is "I'm good at customer service." Based on this information, his communication skills are evaluated according to the following criteria, and a pass / fail decision is made.

[1038] Available working hours: 20 hours or more per week

[1039] Related experience: 1+ years

[1040] Communication skills: High

[1041] Question list:

[1042] 1. "How many hours per week can you work?"

[1043] 2. "What is your previous work experience like?"

[1044] 3. "Please tell us about yourself."

[1045] Based on these questions, analyze Yamada Taro's answers and determine whether he passed or failed.

[1046] Although a detailed description of the present invention has been given, it will be apparent to those skilled in the art that various changes and modifications can be made thereto without departing from the spirit and scope of the invention.

[1047] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1048] Step 1:

[1049] Input: Applicants use a terminal to enter information such as their name, available hours, relevant experience, and personal information into the application.

[1050] How it works: Data entered on the terminal is sent to the server via a REST API through a front-end form.

[1051] Output: The server stores the received applicant information in a database. Each applicant is assigned a unique ID.

[1052] Step 2:

[1053] Input: The server receives the applicant information sent from the terminal.

[1054] How it works: The server uses Node.js and Express to structure the data it receives and store it in a MySQL database.

[1055] Output: The saved data can be accessed from the management screen, and applicant information can be managed centrally.

[1056] Step 3:

[1057] Input: Receive notifications about interview dates and times.

[1058] How it works: The server uses the calendar functionality within the application to send interview date and time reminders to the applicant.

[1059] Output: The applicant receives a notification and accesses the interview system at the specified date and time.

[1060] Step 4:

[1061] Input: The applicant accesses the system at the designated interview date and time and begins a dialogue with the AI ​​interviewer.

[1062] How it works: An AI interviewer (GPT-4) asks applicants questions based on a pre-defined list, and their answers are recorded in real time.

[1063] Output: The applicant's answers are immediately received by the AI ​​interviewer and sent to the server for analysis.

[1064] Step 5:

[1065] Input: The server receives the applicant's response data received from the AI ​​interviewer.

[1066] How it works: The server uses Python for natural language processing to analyze applicants' responses, evaluate their communication skills and other criteria, and calculate an overall score.

[1067] Output: Scores and analysis results are stored in a database.

[1068] Step 6:

[1069] Input: The analysis results and overall score are saved on the server.

[1070] Operation: The server determines whether the applicant passes or fails based on this data, and generates and saves the next procedure information for those who pass.

[1071] Output: A pass / fail result is prepared.

[1072] Step 7:

[1073] Input: The pass / fail result is saved on the server.

[1074] Operation: The server sends the result of the application to the applicant via the notification system. The result details can also be viewed within the applicant's application.

[1075] Output: Applicants receive their results through the application and receive instructions on next steps.

[1076] Example prompt sentence:

[1077] An applicant named "Yamada Taro" exists in the recruitment interview system. His available working hours are "20 hours per week," his related experience is "2 years," and his self-promotional statement is "I'm good at customer service." Based on this information, his communication skills are evaluated according to the following criteria, and a pass / fail decision is made.

[1078] Available working hours: 20 hours or more per week

[1079] Related experience: 1+ years

[1080] Communication skills: High

[1081] Question list:

[1082] 1. "How many hours per week can you work?"

[1083] 2. "What is your previous work experience like?"

[1084] 3. "Please tell us about yourself."

[1085] Based on these questions, analyze Yamada Taro's answers and determine whether he passed or failed.

[1086] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1087] The present invention combines an emotion engine with an AI-based job interview system to further improve the fairness, impartiality, and accuracy of interview evaluations. This system includes a terminal for inputting applicant information, a server for receiving and processing applicant information, a database for storing pass / fail criteria, an AI interviewer who interacts with applicants, a means for recording and analyzing the interaction, and a means for notifying the results. The emotion engine of the present invention also analyzes the user's emotional state and reflects it in the interview questions and evaluations.

[1088] Collection of applicant information

[1089] First, the user (applicant) uses a device (e.g., PC, tablet, smartphone) to enter information such as their name, available working hours, relevant experience, self-promotion, etc. The information sent from the device is received by the server and stored in a database.

[1090] Setting pass / fail criteria

[1091] The store sets the criteria for acceptance through the management screen, including communication skills, available working hours, relevant experience, etc. The criteria entered by the store is received by the server and stored in the database.

[1092] Preparing for and conducting interviews

[1093] The server prepares the interview system based on applicant information and pass / fail criteria. The user accesses the system via their terminal at the designated interview date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue (questions and answers) is recorded in real time by the server. The AI ​​interviewer asks questions based on a pre-set list of questions, and the user answers them.

[1094] Emotion Engine Operation

[1095] The emotion engine connected to the server analyzes the user's voice tone, facial expressions, and language patterns to determine their emotional state. For example, it detects whether the user's voice sounds tense or relaxed. Based on the user's emotional state, the AI ​​interviewer dynamically adjusts the questions and responses it asks. For example, if the user is nervous, it can add questions to relax them.

[1096] Evaluation of interview results

[1097] The server evaluates the user's responses based on the recorded dialogue log and the results of emotion analysis by the emotion engine. It checks whether the available working hours meet the criteria and whether the relevant experience is appropriate. It evaluates communication skills using natural language processing, and also takes into account the results of emotion analysis by the emotion engine. This calculates an overall score and determines whether the candidate passes or fails.

[1098] Notification of results

[1099] After the pass / fail decision is complete, the server notifies the applicant of the result, for example by sending an email to the user with detailed instructions on what to do if they pass. The user receives their result and confirms the next steps.

[1100] As a specific example, the following flow can be considered.

[1101] 1. Entering applicant information: The user uses a terminal to enter the name "Yamada Taro," available working hours "20 hours per week," related experience "2 years," and self-promotion "I'm good at customer service."

[1102] 2. Receiving and storing: The application information sent from the device is received by the server and stored in a database.

[1103] 3. Setting standards: The store sets the available working hours as "20 hours or more per week," related experience as "1 year or more," and communication skills as "high" on the management screen.

[1104] 4. Conducting the interview: The user logs in to the system on the date and time of the interview and interacts with the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?", and the user replies, "I can work 20 hours per week."

[1105] 5. Emotion analysis: The server's emotion engine detects whether the user is nervous from their tone of voice and facial expressions, and changes the AI ​​interviewer's questions to make them more relaxed.

[1106] 6. Evaluation of results: The server evaluates the dialogue log and the emotion analysis results, calculates an overall score, and determines whether the user passes or fails.

[1107] 7. Notification of result: The server creates a success notification email and sends it to the user. The user receives the success notification and confirms the next steps.

[1108] As described above, the present invention further promotes automation and efficiency of interviews, and enables more precise evaluation and response that takes into account the user's emotional state. This allows stores to conduct efficient and fair recruitment activities.

[1109] The processing flow will be explained below.

[1110] Step 1:

[1111] The store logs in to the management screen and enters the pass / fail criteria (e.g., communication skills, available working hours, related experience, etc.).

[1112] Step 2:

[1113] The terminal transmits the entered pass / fail criteria to the server.

[1114] Step 3:

[1115] The server saves the received pass / fail criteria in the database. After saving is complete, a confirmation message is displayed to the store.

[1116] Step 4:

[1117] The user enters information such as name, available working hours, relevant experience, and self-promotion into the application form.

[1118] Step 5:

[1119] The terminal transmits the input application information to the server.

[1120] Step 6:

[1121] The server receives the application information and saves it in the database. After saving is complete, a confirmation message is displayed to the user confirming the application.

[1122] Step 7:

[1123] The server sets the interview date and time and sends a reminder email to the user.

[1124] Step 8:

[1125] The user logs in to the interview system through a terminal at the designated interview date and time.

[1126] Step 9:

[1127] The server starts the interview system and the AI ​​interviewer starts asking questions based on a pre-set list of questions.

[1128] Step 10:

[1129] The user answers questions posed by the AI ​​interviewer, and the answers are sent to the server in real time and recorded.

[1130] Step 11:

[1131] An emotion engine connected to the server analyzes the user's voice tone, facial expressions, and language patterns. For example, if a user answers, "I'm a little nervous, but I want to try this job," the emotion engine will detect nervousness from the user's voice tone and facial expressions.

[1132] Step 12:

[1133] The emotion engine determines the user's emotional state based on the analysis results, generating a result such as "I'm nervous."

[1134] Step 13:

[1135] The server receives the results of the sentiment analysis and dynamically adjusts the questions asked by the AI ​​interviewer, for example, changing the question to, "Let's relax a bit. What has been the most rewarding experience you've had so far?"

[1136] Step 14:

[1137] The user answers a new question, and this answer is also sent in real time to the server and recorded.

[1138] Step 15:

[1139] The server evaluates the user's responses based on the dialogue log and the results of emotion analysis by the emotion engine, for example, checking whether the available working hours meet the criteria and whether the relevant experience is appropriate.

[1140] Step 16:

[1141] The server uses natural language processing to evaluate communication skills, and then incorporates the results of the emotion engine analysis into the evaluation to calculate an overall score.

[1142] Step 17:

[1143] The server determines whether the application passes or fails based on the pass / fail criteria.

[1144] Step 18:

[1145] The server stores the pass / fail results in a database.

[1146] Step 19:

[1147] The server creates a notification email containing the pass / fail result and sends it to the user.

[1148] Step 20:

[1149] The user receives a notification email confirming the result, for example, "Congratulations, you passed!" along with instructions on what to do next.

[1150] The above are the detailed processing steps that combine the emotion engine to collect applicant information, evaluate the interview, and notify the final result. This flow realizes automation and efficiency of interviews, and enables more precise evaluation and response that takes into account the user's emotional state.

[1151] Example 2

[1152] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1153] Conventional recruitment interview systems are often influenced by the interviewer's subjectivity and emotions, which can compromise fairness and impartiality. Furthermore, there is a lack of means to properly grasp the applicant's emotional state, which can lead to a decline in the quality of responses and questions during the interview process. Furthermore, there is no system in place to improve the accuracy of evaluations, resulting in a lack of reliability in hiring decisions. There is a need to solve these problems and conduct fair and efficient recruitment activities.

[1154] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a terminal for inputting applicant information, means for receiving applicant information transmitted from the terminal, a database connected to the server and means for storing pass / fail criteria, an AI system connected to the server and means for engaging in a dialogue with the applicant, and an emotion analysis device connected to the server and means for analyzing the user's emotional state, means for recording the dialogue and making an evaluation, and means for making a pass / fail decision based on the evaluation and notifying the applicant of the result. This enables fair and reliable hiring decisions that take into account the applicant's emotional state.

[1155] A "terminal for inputting applicant information" is a device that allows a user to input their own personal information and job-related data, and includes a PC, tablet, smartphone, etc.

[1156] A "server" is a computer system that receives and sends data over a network and manages and controls the operation of databases and various applications.

[1157] "Database" means a system for efficiently managing and accessing recorded information, including the means for storing applicant information and acceptance criteria.

[1158] "Means for storing acceptance criteria" refers to a data storage system for recording and managing the criteria for employment, including criteria such as available working hours and relevant experience.

[1159] An "AI system" is a system that uses artificial intelligence technology to automatically perform specific tasks and analyses, and includes a means for interacting with applicants.

[1160] An "emotion analysis device" is a device that analyzes a user's voice tone, facial expressions, language patterns, etc. to determine their emotional state.

[1161] "Means for recording and evaluating the dialogue" refers to software or hardware for recording the dialogue between the applicant and the AI ​​system and evaluating that content.

[1162] The "means for determining whether an applicant passes or fails and notifying the applicant of the result" is a system for determining whether an applicant passes or fails based on the content of the dialogue and the evaluation results, and notifying the applicant of the result.

[1163] This invention is a system that improves fairness and evaluation accuracy by combining emotion analysis technology with an AI-based recruitment interview system. This system consists of the following main hardware and software:

[1164] System Configuration

[1165] 1. Terminal: A device on which users enter their application information and interact with the AI ​​interviewer. Terminals include PCs, tablets, smartphones, etc.

[1166] 2. Server: A central computer system that receives, processes, and stores application information, and also integrates and manages AI systems and emotion analysis devices.

[1167] 3. Database: A data storage system for storing applicant information and acceptance criteria.

[1168] 4. AI System: An artificial intelligence-based system for interacting with applicants.

[1169] 5. Emotion analysis device: A device that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[1170] Collection of applicant information

[1171] The user uses the terminal to input information such as name, available working hours, related experience, and self-promotion. For example, the user might input "Yamada Taro," "Available 20 hours per week," "2 years of related experience," and "I'm good at customer service." The input information is sent from the terminal to the server and stored in a database.

[1172] Setting pass / fail criteria

[1173] The store sets the pass / fail criteria for the available working hours, related experience, communication skills, etc. through the management screen. For example, they can set "20 hours or more per week," "1 year or more of related experience," and "high communication skills." These criteria are sent from the terminal to the server and stored in a database.

[1174] Preparing for and conducting interviews

[1175] The server prepares the interview system based on the applicant information and pass / fail criteria. The user accesses the system from their device at the specified date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue is recorded on the server in real time. For example, the AI ​​interviewer asks, "How many hours per week can you work?" and the user replies, "I can work 20 hours per week."

[1176] Emotion analysis

[1177] The emotion analyzer analyzes the user's tone of voice and facial expressions to determine their emotional state. For example, if the AI ​​interviewer detects that the user is nervous, it will add questions to help them relax. It dynamically changes the questions to include, "Is there anything you can do to relax?"

[1178] Evaluation of interview results

[1179] The server comprehensively evaluates the user's responses based on the recorded dialogue log and the results of sentiment analysis. It checks whether the user's available working hours and related experience meet the criteria, and evaluates their communication skills using natural language processing. For example, a user with strong communication skills will respond calmly and clearly.

[1180] Notification of results

[1181] After the pass / fail decision is complete, the server notifies the applicant of the result. For example, if the applicant passes, an email is sent to the user saying, "Congratulations! Here are the detailed steps to proceed to the next step." The user receives the notification and confirms the next steps.

[1182] This system automates the hiring process and enables precise evaluations that take into account the emotional state of applicants, allowing stores to conduct hiring activities efficiently and fairly.

[1183] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1184] Step 1:

[1185] The user uses a terminal to input application information. The input information includes name, available working hours, related experience, self-promotion, etc. For example, the user might input "Yamada Taro," "Available 20 hours per week," "2 years of related experience," and "I'm good at customer service." The input data is temporarily saved on the terminal.

[1186] Input: Application information entered by the user into the device

[1187] Output: Application information temporarily saved on the device

[1188] Step 2:

[1189] The terminal sends the application information to the server. The terminal formats the input data and sends it to the server in the form of an HTTP request or similar. The server receives this request.

[1190] Input: Application information temporarily saved on your device

[1191] Output: Application information sent to the server

[1192] Step 3:

[1193] The server stores the received application information in a database. During this process, the server uses a database management system to check the integrity of the data and store the applicant information securely.

[1194] Input: Application information sent to the server

[1195] Output: Application information stored in a database

[1196] Step 4:

[1197] The store sets the pass / fail criteria through the management screen. For example, they can input criteria such as available working hours of "20 hours or more per week," related experience of "1 year or more," and communication skills of "high." The input criteria are sent from the terminal to the server.

[1198] Input: Pass / fail criteria entered by the store into the terminal

[1199] Output: Pass / fail criteria sent to the server

[1200] Step 5:

[1201] The server stores the received pass / fail criteria in a database, at which point the server checks the integrity of the data and adds the criteria to the database.

[1202] Input: Pass / fail criteria sent to the server

[1203] Output: Pass / fail criteria stored in the database

[1204] Step 6:

[1205] As the interview date and time approaches, the server configures the interview system based on the applicant's information and the criteria for acceptance or rejection, including the list of questions for the AI ​​interviewer and the interview schedule.

[1206] Input: Application information and acceptance criteria stored in the database

[1207] Output: Interview system settings

[1208] Step 7:

[1209] The user accesses the system through a terminal at the specified date and time and begins a conversation with the AI ​​interviewer. The terminal sends a connection request to the server, and the server launches the AI ​​interviewer.

[1210] Input: User access request

[1211] Output: Launched AI interviewer

[1212] Step 8:

[1213] The AI ​​interviewer asks questions to the applicant, and the user answers. This dialogue is sent to the server in real time. For example, the AI ​​interviewer asks, "How many hours can you work per week?" and the user replies, "I can work 20 hours per week."

[1214] Input: User's answer

[1215] Output: Real-time conversation logs recorded on the server

[1216] Step 9:

[1217] The emotion analyzer analyzes the user's tone of voice and facial expressions to determine their emotional state. For example, if the server detects that the user is nervous, it will dynamically change the AI ​​interviewer's questions to make them more relaxed.

[1218] Input: User's voice tone and facial expression data

[1219] Output: Emotional state analysis result

[1220] Step 10:

[1221] The server evaluates the user's responses based on the recorded dialogue logs and sentiment analysis results. This evaluation involves determining whether the user's available working hours and relevant experience meet the criteria, and using natural language processing to evaluate their communication skills.

[1222] Input: Dialogue logs and sentiment analysis results

[1223] Output: Evaluation results and overall score

[1224] Step 11:

[1225] After the pass / fail decision is complete, the server notifies the applicant of the result. For example, if the applicant passes, an email is sent to the user saying, "Congratulations! Here are the detailed steps to proceed to the next step." The user can then check the result on their device.

[1226] Input: Evaluation results and overall score

[1227] Output: Result email sent to applicant

[1228] (Application example 2)

[1229] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1230] Conventional recruitment interview systems did not take into account the emotional state of applicants during evaluation, resulting in problems with the fairness and impartiality of the interview. Furthermore, they were unable to analyze the emotional state of passengers in real time and optimize the in-car environment accordingly. This resulted in passengers' riding experiences not always being satisfactory. Therefore, the challenge is to provide a recruitment interview system that enables emotional analysis of applicants and provides fair and precise evaluations, as well as a system that provides a comfortable riding experience that reflects the emotional state of passengers.

[1231] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1232] In this invention, the server includes a terminal for inputting applicant information, a server that receives the applicant information transmitted from the terminal, a database connected to the server and having means for storing pass / fail criteria, and an interview system including an AI interviewer connected to the server and having means for engaging in a dialogue with the applicant, means for recording the dialogue and making an evaluation, means for analyzing the emotional state of passengers and dynamically adjusting the in-car environment, and means for making a pass / fail decision based on the evaluation and notifying the applicant of the result. This enables fair and precise evaluation that takes into account the emotional state of the applicant, and also enables the in-car environment to be optimized based on the emotional state of passengers, resulting in a more satisfying riding experience.

[1233] A "terminal" is a device through which applicants or customers enter information.

[1234] "Server" means a central computer group that receives and processes information sent from the Terminals.

[1235] "Database" means a storage device for storing pass / fail criteria and other relevant information.

[1236] An "AI interviewer" is software based on artificial intelligence that is used to interact with applicants.

[1237] The "interview system" is a set of mechanisms that includes an AI interviewer to interact with applicants, record the interaction, and evaluate it.

[1238] "Means for conducting evaluation" means a method or device for evaluating the abilities and qualifications of an applicant based on the dialogue records or other criteria.

[1239] The "means for determining whether an applicant passes or fails and notifying the applicant of the result" refers to a method or device for determining whether an applicant passes or fails based on the evaluation and notifying the applicant of the result.

[1240] The "means for analyzing emotional state" refers to a method or apparatus that analyzes voice tone and facial expressions to determine the user's emotions.

[1241] The "means for dynamically adjusting the in-car environment" refers to a method or device for automatically changing the background music, temperature, lighting, seat angle, etc. in the car based on the user's emotional state.

[1242] "Riding customers" are people riding in autonomous vehicles.

[1243] To implement this invention, a terminal for inputting applicant information is first prepared. The terminal can be a smartphone, tablet, or PC. The applicant uses the terminal to input information such as their name, available working hours, relevant experience, and self-promotion. The input information is sent to a server, which receives it and stores it in a database.

[1244] Next, the server receives the pass / fail criteria from the administrator and stores them in the database. The administrator sets the criteria such as available working hours, relevant experience, communication ability, etc. The pass / fail criteria are important indicators used to evaluate the job interview.

[1245] The server prepares an AI interviewer based on applicant information and pass / fail criteria. The AI ​​interviewer uses natural language processing technology to converse with the applicant. During the conversation, the server activates an emotion engine that analyzes the applicant's tone of voice and facial expressions to understand the applicant's emotional state. The emotion engine is built using software such as TensorFlow and OpenCV. For example, it can add questions to relax nervous applicants.

[1246] After the conversation is over, the server evaluates the applicant based on the content of the conversation and the results of sentiment analysis. Using natural language processing technology, the server evaluates the applicant's communication skills and calculates an overall score based on criteria such as available working hours and relevant experience. Based on this, the server determines whether the applicant is successful and notifies the applicant of the result.

[1247] The system also has the functionality to be used in autonomous vehicles. It can analyze the emotional state of passengers and dynamically adjust the in-car environment. Using the smartphone's camera and microphone, it analyzes the passenger's facial expressions and tone of voice and automatically adjusts the in-car background music, temperature, lighting, seat angle, and more, providing the optimal experience for passengers while they are in the vehicle.

[1248] Examples:

[1249] After a user gets into an autonomous vehicle, they launch the "Emotion Reader" app. The app uses the smartphone's camera and microphone to analyze the user's emotional state in real time. For example, if the app detects that the user's facial expression indicates "surprise" or "tension," it automatically adjusts the in-car background music to relaxing music. Furthermore, after a long ride, the app reports any changes in the user's emotions (from tension to relaxation).

[1250] Hardware and software used:

[1251] Smartphone: A device equipped with a camera and microphone to analyze the emotional state of passengers.

[1252] Python: The primary programming language used.

[1253] OpenCV: Used for real-time analysis of camera footage.

[1254] TensorFlow: Used to build the sentiment analysis model.

[1255] Firebase: Used for database and real-time updates.

[1256] Flask: A microframework for providing a sentiment analysis API on the backend.

[1257] Example prompt sentence:

[1258] "Tell me about a system in which a smartphone app inside a self-driving vehicle uses cameras and microphones to analyze passenger emotions in real time and optimize the ride experience (background music, temperature, lighting)."

[1259] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1260] Step 1:

[1261] Input: The applicant enters their name, available working hours, relevant experience, and self-promotional information into the terminal.

[1262] Output: The entered information is sent to the server.

[1263] Specific operation: The user enters the required information into the form on the device and presses the submit button. The device converts the entered information into JSON format and sends it to the server. The server saves the received information in a database in real time.

[1264] Step 2:

[1265] Input: The administrator inputs the pass / fail criteria from the terminal.

[1266] Output: The pass / fail criteria is sent to the server and stored in a database.

[1267] Specific operation: The administrator sets criteria such as available working hours, relevant experience, and communication skills on the management screen and sends them to the server, which receives them and stores them in the database.

[1268] Step 3:

[1269] Input: The server obtains applicant information and acceptance criteria.

[1270] Output: The AI ​​interviewer is prepared.

[1271] Specific operation: The server retrieves applicant information and pass / fail criteria from the database, and prepares an AI interviewer based on that information. The AI ​​interviewer loads a pre-set list of questions.

[1272] Step 4:

[1273] Input: The applicant uses the terminal to access the interview system.

[1274] Output: A conversation between the AI ​​interviewer and the applicant begins.

[1275] Specific operation: The applicant logs in to the system from their terminal at the designated interview date and time. The server authenticates the applicant and starts a dialogue session with the AI ​​interviewer. The dialogue is recorded in real time.

[1276] Step 5:

[1277] Input: AI interviewer analyzes applicant's tone of voice and facial expressions.

[1278] Output: Sentiment analysis results are obtained.

[1279] Specific operation: The server analyzes the camera footage using OpenCV to recognize the applicant's facial expressions, and analyzes the voice data using a TensorFlow model to analyze the voice tone. The analysis results are saved on the server.

[1280] Step 6:

[1281] Input: Dialogue records and sentiment analysis results are collected on the server.

[1282] Output: The applicant's overall rating is calculated.

[1283] How it works: The server uses natural language processing technology to evaluate the applicant's communication skills based on the dialogue log and sentiment analysis results, and calculates an overall score based on criteria such as available working hours and related experience.

[1284] Step 7:

[1285] Input: The server will determine whether the candidate passes or fails based on the overall evaluation.

[1286] Output: The applicant is notified of the results.

[1287] Specific operation: The server determines whether the applicant has passed or failed based on the overall evaluation score, and notifies the applicant of the result by email. The applicant receives the result and confirms the next steps.

[1288] Step 8:

[1289] Input: A customer enters an autonomous vehicle and launches a smartphone app.

[1290] Output: The in-car environment is dynamically adjusted based on the customer's emotional state.

[1291] How it works: The user launches the "Emotion Reader" smartphone app while in the car. The smartphone's camera and microphone are used to analyze the customer's facial expressions and tone of voice in real time. Based on the analysis results, the car's background music, temperature, lighting, seat angle, and other settings are automatically adjusted.

[1292] Example prompt sentence:

[1293] "Tell me about a system in which a smartphone app inside a self-driving vehicle uses cameras and microphones to analyze passenger emotions in real time and optimize the ride experience (background music, temperature, lighting)."

[1294] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1295] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1296] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1297] [Fourth embodiment]

[1298] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1299] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1300] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1301] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1302] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1303] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1304] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1305] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1306] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1307] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1308] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1309] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1310] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1311] This invention is a recruitment interview system that uses AI, which reduces the time and effort required for job interviews at stores and realizes a fair and efficient recruitment process. The system includes a terminal for inputting applicant information, a server for receiving and processing applicant information, a database for storing pass / fail criteria, an AI interviewer who interacts with applicants, a means for recording and analyzing the interaction, and a means for notifying the results.

[1312] Collection of applicant information

[1313] First, the user (applicant) uses a device (e.g., PC, tablet, smartphone) to enter the necessary information into the application form. This information includes name, available working hours, relevant experience, self-promotion, etc. The information sent from the device is received by the server and stored in a database.

[1314] Setting pass / fail criteria

[1315] The store sets various criteria for acceptance through the management screen, such as communication skills, available working hours, relevant experience, etc. The criteria entered by the store are received by the server and stored in the database.

[1316] Preparing for and conducting interviews

[1317] The server prepares the interview system based on applicant information and pass / fail criteria. The user accesses the system via their device at the designated interview date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue (questions and answers) is recorded in real time by the server. The AI ​​interviewer asks questions based on a pre-set list of questions and accepts the user's answers.

[1318] Evaluation of interview results

[1319] The server analyzes the recorded dialogue logs and evaluates the user's responses. For example, it checks whether the available working hours meet the criteria and whether the relevant experience is appropriate. It also evaluates the user's communication skills using natural language processing. Based on these evaluations, the server calculates an overall score and determines whether the user is accepted or rejected.

[1320] Notification of results

[1321] After the pass / fail decision is complete, the server notifies the applicant of the result, for example by sending an email to the applicant with details of the next steps if they pass. The user receives their result and can confirm the next steps.

[1322] As a specific example, the following flow can be considered.

[1323] 1. Entering applicant information: The user uses a terminal to enter the name "Yamada Taro," available working hours "20 hours per week," related experience "2 years," and self-promotion "I'm good at customer service."

[1324] 2. Receiving and storing: The application information sent from the device is received by the server and stored in a database.

[1325] 3. Setting standards: The store sets the available working hours as "20 hours or more per week," related experience as "1 year or more," and communication skills as "high" on the management screen.

[1326] 4. Conducting the interview: The user logs in to the system on the date and time of the interview and interacts with the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?", and the user replies, "I can work 20 hours per week."

[1327] 5. Evaluation of results: The server analyzes the dialogue log, compares the user's available working hours and related experience with the standard, and evaluates their communication skills using natural language processing.

[1328] 6. Pass / fail determination and notification: The server calculates the applicant's overall score, determines whether they passed or failed, and notifies the user of the result by email.

[1329] The above steps allow stores to evaluate applicants and conduct hiring activities efficiently and fairly, and the system can significantly reduce the time and effort required for interviews.

[1330] The processing flow will be explained below.

[1331] Step 1:

[1332] The store logs in to the management screen and enters the pass / fail criteria (e.g., communication skills, available working hours, related experience, etc.).

[1333] Step 2:

[1334] The terminal transmits the entered pass / fail criteria to the server.

[1335] Step 3:

[1336] The server saves the received pass / fail criteria in the database. After saving is complete, a confirmation message is displayed to the store.

[1337] Step 4:

[1338] The user enters information such as name, available working hours, relevant experience, and self-promotion into the application form.

[1339] Step 5:

[1340] The terminal transmits the input application information to the server.

[1341] Step 6:

[1342] The server receives the application information and saves it in the database. After saving is complete, a confirmation message is displayed to the user confirming the application.

[1343] Step 7:

[1344] The server sets the interview date and time and sends a reminder email to the user.

[1345] Step 8:

[1346] The user logs in to the interview system through a terminal at the designated interview date and time.

[1347] Step 9:

[1348] The server starts the interview system and the AI ​​interviewer starts asking questions based on a pre-set list of questions.

[1349] Step 10:

[1350] The user answers questions posed by the AI ​​interviewer, and the answers are sent to the server in real time and recorded.

[1351] Step 11:

[1352] The server analyzes the recorded dialogue log and evaluates the user's responses.

[1353] Step 12:

[1354] The server calculates the evaluation results based on the pass / fail criteria and determines whether the application passes or fails.

[1355] Step 13:

[1356] The server stores the pass / fail results in a database.

[1357] Step 14:

[1358] The server creates a notification email containing the pass / fail result and sends it to the user.

[1359] Step 15:

[1360] The user receives a notification email and checks the results.

[1361] The above are the detailed process steps from collecting applicant information to interview evaluation and final notification of results. This flow realizes automation and efficiency of interviews.

[1362] Example 1

[1363] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1364] The existing recruitment interview process requires a great deal of time and effort, and the interviewer's subjectivity can affect the results. There are multiple problems, such as unfair judgments and reduced efficiency due to busy schedules. Furthermore, inconsistent evaluations of applicants make it difficult to select the best candidates. This invention aims to realize a fair and efficient recruitment process.

[1365] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1366] In this invention, the server includes a means for receiving applicant information and storing it in a database, a means for analyzing the recorded dialogue log and evaluating the applicant using natural language processing, and a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result, thereby making it possible to improve the efficiency and fairness of the job interview process.

[1367] "Terminal" means an electronic device used by a user to input and transmit information.

[1368] "Server" means the central system that receives, processes and manages information sent from the terminals.

[1369] A "database" is a storage device within the system that stores and manages data such as applicant information and acceptance criteria.

[1370] The "AI Interviewer" is an artificial intelligence system that interacts with applicants and collects and evaluates their responses based on questions.

[1371] The "interview system" refers to the entire system, including the AI ​​interviewer, that executes and manages the interview process with applicants.

[1372] A "dialogue log" is data that records the content of the conversation between an AI interviewer and an applicant during an interview.

[1373] "Natural language processing" is a technology that allows computers to understand, analyze, and evaluate human language.

[1374] "Pass / fail assessment" is the process of determining whether an applicant passes or fails based on the results of their evaluation.

[1375] "Notification means" refers to the means for informing applicants of the results of their application, such as sending an email.

[1376] In the embodiment for carrying out the invention, this system is a recruitment interview system using AI, which reduces the time and effort required for recruitment interviews on the store side and realizes a fair and efficient recruitment process. This system includes the following elements.

[1377] Collection of applicant information

[1378] The user uses a device (PC, tablet, smartphone, etc.) to open a browser and access the application form. In the application form, they enter information about the applicant, such as their name, available working hours, relevant experience, and self-promotion. The device sends the information entered by the user in JSON format to the server, which then stores the information in a database.

[1379] Setting pass / fail criteria

[1380] The store uses a management terminal to access the management screen and input the pass / fail criteria, such as available working hours, communication skills, relevant experience, etc. The criteria sent from the management screen are received by the server and stored in a database.

[1381] Preparing for and conducting interviews

[1382] The server prepares the interview system based on the applicant's information and pass / fail criteria. At the designated interview date and time, the user accesses the system through their device and begins a dialogue with the AI ​​interviewer. The AI ​​interviewer asks questions from a pre-set list. For example, they may ask, "How many hours per week can you work?" The user responds, and the device sends the answers to the server, which records them in real time.

[1383] Evaluation of interview results

[1384] After the interview, the server analyzes the recorded dialogue log and evaluates the applicant's communication skills using natural language processing. It also checks whether the applicant's answers meet the criteria for acceptance or rejection. For example, it checks whether the applicant's available working hours and related experience meet the criteria. Based on these evaluations, it calculates an overall score and determines whether the applicant is accepted or rejected.

[1385] Notification of results

[1386] After the pass / fail decision is complete, the server notifies the applicant of the result by email. For example, if the applicant passes, an email containing details of the next steps is sent. The user opens the received email, checks the result, and proceeds to the next step.

[1387] For example, consider the following prompt:

[1388] "Please evaluate the applicant's answers based on whether they are available to work 20 hours or more per week, have at least one year of related experience, and have strong communication skills, and calculate an overall score. Based on this score, you will decide whether the applicant has passed or failed, and if successful, you will send an email with details of the next steps."

[1389] The above steps allow stores to evaluate applicants and conduct hiring activities efficiently and fairly, and the system can significantly reduce the time and effort required for interviews.

[1390] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1391] Step 1:

[1392] The user displays the input screen. The user opens a browser on a device (PC, tablet, smartphone, etc.) and accesses the application form. Input: Browser URL Output: Application form screen

[1393] Step 2:

[1394] The user enters applicant information. The user enters the name, available working hours, related experience, self-promotion, etc. into the application form. Input: Applicant information (name, available working hours, related experience, self-promotion) Output: Completed application form

[1395] Step 3:

[1396] The terminal sends applicant information to the server. When the user presses the "Send" button, the terminal sends the entered information to the server in JSON format. Input: Applicant information (JSON format) Output: Data sent to the server

[1397] Step 4:

[1398] The server receives the applicant information and saves it in the database. The server saves the received information in the database and returns a response to the terminal indicating that saving has been completed. Input: Applicant information (JSON format) Output: Saved in the database, save completion response

[1399] Step 5:

[1400] The store displays the management screen. The store accesses the management screen using a management terminal. Input: Browser URL Output: Management screen

[1401] Step 6:

[1402] The store enters the pass / fail criteria. From the management screen, the store enters pass / fail criteria such as available working hours, communication skills, and related experience. Input: Available working hours, communication skills, and related experience criteria Output: Entered pass / fail criteria

[1403] Step 7:

[1404] The terminal sends the pass / fail criteria to the server. When the store presses the "Save" button, the terminal sends the criteria to the server in JSON format. Input: Pass / fail criteria (JSON format) Output: Data sent to the server

[1405] Step 8:

[1406] The server saves the pass / fail criteria in the database. The server saves the received criteria in the database and returns a response to the terminal indicating that saving is complete. Input: Pass / fail criteria (JSON format) Output: Saved in the database, save completion response

[1407] Step 9:

[1408] The server prepares the interview system. Based on the applicant information and pass / fail criteria, the server configures the interview system. Input: Applicant information, pass / fail criteria Output: Interview system is ready

[1409] Step 10:

[1410] The user accesses the system at the specified interview date and time. The user logs in to the interview system through a terminal. Input: Interview login URL Output: Interview screen

[1411] Step 11:

[1412] The AI ​​interviewer asks questions. The AI ​​interviewer on the server selects questions from a pre-set list and presents them to the user. Input: Question list Output: Questions asked by the AI ​​interviewer

[1413] Step 12:

[1414] The user answers the question. The user answers the question by text or voice. Input: User's answer Output: Answer data

[1415] Step 13:

[1416] The server records the answers in real time. The server receives the user's answers in real time and saves them as a dialogue log. Input: Answer data Output: Dialogue log

[1417] Step 14:

[1418] The server analyzes the dialogue log. After the interview, the server analyzes the recorded dialogue log and evaluates the applicant's communication skills and responses. Input: Dialogue log Output: Analysis results

[1419] Step 15:

[1420] The server determines whether the candidate passes or fails based on the evaluation results. The server calculates a total score based on criteria such as communication ability, available working hours, and related experience, and determines whether the candidate passes or fails. Input: Analysis results, pass / fail criteria Output: Pass / fail decision

[1421] Step 16:

[1422] The server will notify the result. The server will send the result of the pass / fail decision to the applicant by email. Input: Pass / fail decision result Output: Notification email

[1423] Step 17:

[1424] The user checks the results. The user opens the received email and checks the pass / fail result and next steps. Input: Notification email Output: Check pass / fail result

[1425] (Application example 1)

[1426] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1427] The traditional hiring process had issues such as time-consuming scheduling between applicants and interviewers, and subjective evaluation of applicants. Additionally, conducting interviews and notifying interviewers of results was time-consuming, making efficient hiring difficult. The present invention aims to solve these problems by utilizing AI, realizing a fairer and more efficient hiring process.

[1428] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1429] In this invention, the server includes a terminal for inputting applicant information, a means for receiving applicant information sent from the terminal, a database connected to the server and a means for storing pass / fail criteria, and an interview system including an AI interviewer connected to the server, which includes a means for engaging in dialogue with the applicant, a means for recording the dialogue between the AI ​​interviewer and the applicant and evaluating them in real time, a means for making a pass / fail decision based on the evaluation and notifying the applicant of the result via an application, and a means for notifying the applicant of the pass / fail decision and providing information on next steps. This automates the job interview process, enabling efficient and fair recruitment activities. Furthermore, using a generative AI model to evaluate applicants improves the reliability and accuracy of the evaluation.

[1430] "Terminal" means the device used by the applicant to enter information, including smartphones, computers, tablets, etc.

[1431] A "server" is a computer system that receives and processes applicant information sent from a terminal.

[1432] "Database" refers to an information storage system for storing and managing applicant information and acceptance criteria.

[1433] "Pass / fail criteria" are criteria used to evaluate applicants during recruitment, and include working hours, related experience, communication skills, etc.

[1434] The "AI Interviewer" is a virtual interviewer that uses artificial intelligence to converse with applicants and analyze and evaluate the content of the conversation.

[1435] "Recording" means saving the conversation between the AI ​​interviewer and the applicant in digital format.

[1436] "Real-time" means that processing and evaluation occurs immediately during the conversation with the applicant.

[1437] "Evaluation" is the act of judging an applicant's suitability and ability based on the applicant's answers and the content of the dialogue.

[1438] "Pass / fail decision" refers to determining whether an applicant meets the employment criteria based on the evaluation results.

[1439] "Notification of results" refers to the act of informing applicants of the results of the pass / fail decision.

[1440] An "application" is software that runs on a smartphone or other digital device and is used by applicants to receive results or conduct interviews.

[1441] "Procedural information" refers to specific information provided to successful applicants as the next step, including information on required documents and start dates.

[1442] A "generative AI model" is an artificial intelligence model trained to perform natural language processing and data analysis, and is used to analyze and evaluate applicant responses.

[1443] The present invention is a system for automating the job interview process. This system is designed to perform an entire process from inputting applicant information to notifying the interview results. Detailed embodiments of the present invention are described below.

[1444] Hardware and software used

[1445] 1. Device: The device used by the applicant to enter information, including smartphones, computers, tablets, etc. In particular, we will develop cross-platform mobile applications using React Native.

[1446] 2. Server: This is a computer system that receives and processes applicant information sent from the terminal. The server is built using Node.js and Express and performs data processing.

[1447] 3. Database: An information storage system for storing and managing applicant information and acceptance criteria. MySQL is used.

[1448] 4. AI Interviewer: An artificial intelligence that interacts with applicants and uses generative AI models such as GPT-4 to process interview questions and applicant responses.

[1449] 5. Natural language processing system: Natural language processing is implemented using Python to analyze the applicants' responses.

[1450] 6. Application: This is the software that allows applicants to go through the interview process and notify them of the evaluation results. Applicants participate in the interview and receive the results via the application.

[1451] Data processing and calculation

[1452] The server processes and calculates the data as follows:

[1453] 1. The applicant uses a terminal to enter information such as name, available working hours, relevant experience, and self-promotion into the application. This information is then sent to the server via a REST API.

[1454] 2. The server receives this information and stores it in a MySQL database, where each applicant is assigned a unique ID.

[1455] 3. At the designated interview date and time, the applicant accesses the system through the application and begins a dialogue with the AI ​​interviewer. The AI ​​interviewer asks the applicant questions based on a pre-set list of questions and accepts the applicant's answers.

[1456] 4. The AI ​​interviewer (generative AI model) analyzes the applicant's responses in real time and generates a score for evaluation. For example, it checks whether the applicant's available hours and relevant experience meet the set criteria. Communication skills are also evaluated using a natural language processing system.

[1457] 5. The server makes a comprehensive pass / fail decision based on the analyzed dialogue log. The result of this decision is notified to the application used by the applicant. Successful applicants are then informed of the next steps.

[1458] Specific examples

[1459] As a concrete example, let's say an applicant enters the name "Yamada Taro," his available working hours as "20 hours per week," his related experience as "2 years," and his self-promotion as "I'm good at customer service." This information is sent to the server and saved in the database.

[1460] At the time of the interview, the applicant logs in through the application and speaks to the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?" and the applicant replies, "I can work 20 hours per week." This response is recorded in real time and analyzed by GPT-4. Based on the analysis results, the server makes a pass / fail decision and notifies the applicant of the result.

[1461] Prompt Sentence Examples

[1462] Here are some examples of prompts for generative AI models:

[1463] An applicant named "Yamada Taro" exists in the recruitment interview system. His available working hours are "20 hours per week," his related experience is "2 years," and his self-promotional statement is "I'm good at customer service." Based on this information, his communication skills are evaluated according to the following criteria, and a pass / fail decision is made.

[1464] Available working hours: 20 hours or more per week

[1465] Related experience: 1+ years

[1466] Communication skills: High

[1467] Question list:

[1468] 1. "How many hours per week can you work?"

[1469] 2. "What is your previous work experience like?"

[1470] 3. "Please tell us about yourself."

[1471] Based on these questions, analyze Yamada Taro's answers and determine whether he passed or failed.

[1472] Although a detailed description of the present invention has been given, it will be apparent to those skilled in the art that various changes and modifications can be made thereto without departing from the spirit and scope of the invention.

[1473] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1474] Step 1:

[1475] Input: Applicants use a terminal to enter information such as their name, available hours, relevant experience, and personal information into the application.

[1476] How it works: Data entered on the terminal is sent to the server via a REST API through a front-end form.

[1477] Output: The server stores the received applicant information in a database. Each applicant is assigned a unique ID.

[1478] Step 2:

[1479] Input: The server receives the applicant information sent from the terminal.

[1480] How it works: The server uses Node.js and Express to structure the data it receives and store it in a MySQL database.

[1481] Output: The saved data can be accessed from the management screen, and applicant information can be managed centrally.

[1482] Step 3:

[1483] Input: Receive notifications about interview dates and times.

[1484] How it works: The server uses the calendar functionality within the application to send interview date and time reminders to the applicant.

[1485] Output: The applicant receives a notification and accesses the interview system at the specified date and time.

[1486] Step 4:

[1487] Input: The applicant accesses the system at the designated interview date and time and begins a dialogue with the AI ​​interviewer.

[1488] How it works: An AI interviewer (GPT-4) asks applicants questions based on a pre-defined list, and their answers are recorded in real time.

[1489] Output: The applicant's answers are immediately received by the AI ​​interviewer and sent to the server for analysis.

[1490] Step 5:

[1491] Input: The server receives the applicant's response data received from the AI ​​interviewer.

[1492] How it works: The server uses Python for natural language processing to analyze applicants' responses, evaluate their communication skills and other criteria, and calculate an overall score.

[1493] Output: Scores and analysis results are stored in a database.

[1494] Step 6:

[1495] Input: The analysis results and overall score are saved on the server.

[1496] Operation: The server determines whether the applicant passes or fails based on this data, and generates and saves the next procedure information for those who pass.

[1497] Output: A pass / fail result is prepared.

[1498] Step 7:

[1499] Input: The pass / fail result is saved on the server.

[1500] Operation: The server sends the result of the application to the applicant via the notification system. The result details can also be viewed within the applicant's application.

[1501] Output: Applicants receive their results through the application and receive instructions on next steps.

[1502] Example prompt sentence:

[1503] An applicant named "Yamada Taro" exists in the recruitment interview system. His available working hours are "20 hours per week," his related experience is "2 years," and his self-promotional statement is "I'm good at customer service." Based on this information, his communication skills are evaluated according to the following criteria, and a pass / fail decision is made.

[1504] Available working hours: 20 hours or more per week

[1505] Related experience: 1+ years

[1506] Communication skills: High

[1507] Question list:

[1508] 1. "How many hours per week can you work?"

[1509] 2. "What is your previous work experience like?"

[1510] 3. "Please tell us about yourself."

[1511] Based on these questions, analyze Yamada Taro's answers and determine whether he passed or failed.

[1512] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1513] The present invention combines an emotion engine with an AI-based job interview system to further improve the fairness, impartiality, and accuracy of interview evaluations. This system includes a terminal for inputting applicant information, a server for receiving and processing applicant information, a database for storing pass / fail criteria, an AI interviewer who interacts with applicants, a means for recording and analyzing the interaction, and a means for notifying the results. The emotion engine of the present invention also analyzes the user's emotional state and reflects it in the interview questions and evaluations.

[1514] Collection of applicant information

[1515] First, the user (applicant) uses a device (e.g., PC, tablet, smartphone) to enter information such as their name, available working hours, relevant experience, self-promotion, etc. The information sent from the device is received by the server and stored in a database.

[1516] Setting pass / fail criteria

[1517] The store sets the criteria for acceptance through the management screen, including communication skills, available working hours, relevant experience, etc. The criteria entered by the store is received by the server and stored in the database.

[1518] Preparing for and conducting interviews

[1519] The server prepares the interview system based on applicant information and pass / fail criteria. The user accesses the system via their terminal at the designated interview date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue (questions and answers) is recorded in real time by the server. The AI ​​interviewer asks questions based on a pre-set list of questions, and the user answers them.

[1520] Emotion Engine Operation

[1521] The emotion engine connected to the server analyzes the user's voice tone, facial expressions, and language patterns to determine their emotional state. For example, it detects whether the user's voice sounds tense or relaxed. Based on the user's emotional state, the AI ​​interviewer dynamically adjusts the questions and responses it asks. For example, if the user is nervous, it can add questions to relax them.

[1522] Evaluation of interview results

[1523] The server evaluates the user's responses based on the recorded dialogue log and the results of emotion analysis by the emotion engine. It checks whether the available working hours meet the criteria and whether the relevant experience is appropriate. It evaluates communication skills using natural language processing, and also takes into account the results of emotion analysis by the emotion engine. This calculates an overall score and determines whether the candidate passes or fails.

[1524] Notification of results

[1525] After the pass / fail decision is complete, the server notifies the applicant of the result, for example by sending an email to the user with detailed instructions on what to do if they pass. The user receives their result and confirms the next steps.

[1526] As a specific example, the following flow can be considered.

[1527] 1. Entering applicant information: The user uses a terminal to enter the name "Yamada Taro," available working hours "20 hours per week," related experience "2 years," and self-promotion "I'm good at customer service."

[1528] 2. Receiving and storing: The application information sent from the device is received by the server and stored in a database.

[1529] 3. Setting standards: The store sets the available working hours as "20 hours or more per week," related experience as "1 year or more," and communication skills as "high" on the management screen.

[1530] 4. Conducting the interview: The user logs in to the system on the date and time of the interview and interacts with the AI ​​interviewer. The AI ​​interviewer asks, "How many hours per week can you work?", and the user replies, "I can work 20 hours per week."

[1531] 5. Emotion analysis: The server's emotion engine detects whether the user is nervous from their tone of voice and facial expressions, and changes the AI ​​interviewer's questions to make them more relaxed.

[1532] 6. Evaluation of results: The server evaluates the dialogue log and the emotion analysis results, calculates an overall score, and determines whether the user passes or fails.

[1533] 7. Notification of result: The server creates a success notification email and sends it to the user. The user receives the success notification and confirms the next steps.

[1534] As described above, the present invention further promotes automation and efficiency of interviews, and enables more precise evaluation and response that takes into account the user's emotional state. This allows stores to conduct efficient and fair recruitment activities.

[1535] The processing flow will be explained below.

[1536] Step 1:

[1537] The store logs in to the management screen and enters the pass / fail criteria (e.g., communication skills, available working hours, related experience, etc.).

[1538] Step 2:

[1539] The terminal transmits the entered pass / fail criteria to the server.

[1540] Step 3:

[1541] The server saves the received pass / fail criteria in the database. After saving is complete, a confirmation message is displayed to the store.

[1542] Step 4:

[1543] The user enters information such as name, available working hours, relevant experience, and self-promotion into the application form.

[1544] Step 5:

[1545] The terminal transmits the input application information to the server.

[1546] Step 6:

[1547] The server receives the application information and saves it in the database. After saving is complete, a confirmation message is displayed to the user confirming the application.

[1548] Step 7:

[1549] The server sets the interview date and time and sends a reminder email to the user.

[1550] Step 8:

[1551] The user logs in to the interview system through a terminal at the designated interview date and time.

[1552] Step 9:

[1553] The server starts the interview system and the AI ​​interviewer starts asking questions based on a pre-set list of questions.

[1554] Step 10:

[1555] The user answers questions posed by the AI ​​interviewer, and the answers are sent to the server in real time and recorded.

[1556] Step 11:

[1557] An emotion engine connected to the server analyzes the user's voice tone, facial expressions, and language patterns. For example, if a user answers, "I'm a little nervous, but I want to try this job," the emotion engine will detect nervousness from the user's voice tone and facial expressions.

[1558] Step 12:

[1559] The emotion engine determines the user's emotional state based on the analysis results, generating a result such as "I'm nervous."

[1560] Step 13:

[1561] The server receives the results of the sentiment analysis and dynamically adjusts the questions asked by the AI ​​interviewer, for example, changing the question to, "Let's relax a bit. What has been the most rewarding experience you've had so far?"

[1562] Step 14:

[1563] The user answers a new question, and this answer is also sent in real time to the server and recorded.

[1564] Step 15:

[1565] The server evaluates the user's responses based on the dialogue log and the results of emotion analysis by the emotion engine, for example, checking whether the available working hours meet the criteria and whether the relevant experience is appropriate.

[1566] Step 16:

[1567] The server uses natural language processing to evaluate communication skills, and then incorporates the results of the emotion engine analysis into the evaluation to calculate an overall score.

[1568] Step 17:

[1569] The server determines whether the application passes or fails based on the pass / fail criteria.

[1570] Step 18:

[1571] The server stores the pass / fail results in a database.

[1572] Step 19:

[1573] The server creates a notification email containing the pass / fail result and sends it to the user.

[1574] Step 20:

[1575] The user receives a notification email confirming the result, for example, "Congratulations, you passed!" along with instructions on what to do next.

[1576] The above are the detailed processing steps that combine the emotion engine to collect applicant information, evaluate the interview, and notify the final result. This flow realizes automation and efficiency of interviews, and enables more precise evaluation and response that takes into account the user's emotional state.

[1577] Example 2

[1578] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1579] Conventional recruitment interview systems are often influenced by the interviewer's subjectivity and emotions, which can compromise fairness and impartiality. Furthermore, there is a lack of means to properly grasp the applicant's emotional state, which can lead to a decline in the quality of responses and questions during the interview process. Furthermore, there is no system in place to improve the accuracy of evaluations, resulting in a lack of reliability in hiring decisions. There is a need to solve these problems and conduct fair and efficient recruitment activities.

[1580] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a terminal for inputting applicant information, means for receiving applicant information transmitted from the terminal, a database connected to the server and means for storing pass / fail criteria, an AI system connected to the server and means for engaging in a dialogue with the applicant, and an emotion analysis device connected to the server and means for analyzing the user's emotional state, means for recording the dialogue and making an evaluation, and means for making a pass / fail decision based on the evaluation and notifying the applicant of the result. This enables fair and reliable hiring decisions that take into account the applicant's emotional state.

[1581] A "terminal for inputting applicant information" is a device that allows a user to input their own personal information and job-related data, and includes a PC, tablet, smartphone, etc.

[1582] A "server" is a computer system that receives and sends data over a network and manages and controls the operation of databases and various applications.

[1583] "Database" means a system for efficiently managing and accessing recorded information, including the means for storing applicant information and acceptance criteria.

[1584] "Means for storing acceptance criteria" refers to a data storage system for recording and managing the criteria for employment, including criteria such as available working hours and relevant experience.

[1585] An "AI system" is a system that uses artificial intelligence technology to automatically perform specific tasks and analyses, and includes a means for interacting with applicants.

[1586] An "emotion analysis device" is a device that analyzes a user's voice tone, facial expressions, language patterns, etc. to determine their emotional state.

[1587] "Means for recording and evaluating the dialogue" refers to software or hardware for recording the dialogue between the applicant and the AI ​​system and evaluating that content.

[1588] The "means for determining whether an applicant passes or fails and notifying the applicant of the result" is a system for determining whether an applicant passes or fails based on the content of the dialogue and the evaluation results, and notifying the applicant of the result.

[1589] This invention is a system that improves fairness and evaluation accuracy by combining emotion analysis technology with an AI-based recruitment interview system. This system consists of the following main hardware and software:

[1590] System Configuration

[1591] 1. Terminal: A device on which users enter their application information and interact with the AI ​​interviewer. Terminals include PCs, tablets, smartphones, etc.

[1592] 2. Server: A central computer system that receives, processes, and stores application information, and also integrates and manages AI systems and emotion analysis devices.

[1593] 3. Database: A data storage system for storing applicant information and acceptance criteria.

[1594] 4. AI System: An artificial intelligence-based system for interacting with applicants.

[1595] 5. Emotion analysis device: A device that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[1596] Collection of applicant information

[1597] The user uses the terminal to input information such as name, available working hours, related experience, and self-promotion. For example, the user might input "Yamada Taro," "Available 20 hours per week," "2 years of related experience," and "I'm good at customer service." The input information is sent from the terminal to the server and stored in a database.

[1598] Setting pass / fail criteria

[1599] The store sets the pass / fail criteria for the available working hours, related experience, communication skills, etc. through the management screen. For example, they can set "20 hours or more per week," "1 year or more of related experience," and "high communication skills." These criteria are sent from the terminal to the server and stored in a database.

[1600] Preparing for and conducting interviews

[1601] The server prepares the interview system based on the applicant information and pass / fail criteria. The user accesses the system from their device at the specified date and time and begins a dialogue with the AI ​​interviewer. The content of the dialogue is recorded on the server in real time. For example, the AI ​​interviewer asks, "How many hours per week can you work?" and the user replies, "I can work 20 hours per week."

[1602] Emotion analysis

[1603] The emotion analyzer analyzes the user's tone of voice and facial expressions to determine their emotional state. For example, if the AI ​​interviewer detects that the user is nervous, it will add questions to help them relax. It dynamically changes the questions to include, "Is there anything you can do to relax?"

[1604] Evaluation of interview results

[1605] The server comprehensively evaluates the user's responses based on the recorded dialogue log and the results of sentiment analysis. It checks whether the user's available working hours and related experience meet the criteria, and evaluates their communication skills using natural language processing. For example, a user with strong communication skills will respond calmly and clearly.

[1606] Notification of results

[1607] After the pass / fail decision is complete, the server notifies the applicant of the result. For example, if the applicant passes, an email is sent to the user saying, "Congratulations! Here are the detailed steps to proceed to the next step." The user receives the notification and confirms the next steps.

[1608] This system automates the hiring process and enables precise evaluations that take into account the emotional state of applicants, allowing stores to conduct hiring activities efficiently and fairly.

[1609] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1610] Step 1:

[1611] The user uses a terminal to input application information. The input information includes name, available working hours, related experience, self-promotion, etc. For example, the user might input "Yamada Taro," "Available 20 hours per week," "2 years of related experience," and "I'm good at customer service." The input data is temporarily saved on the terminal.

[1612] Input: Application information entered by the user into the device

[1613] Output: Application information temporarily saved on the device

[1614] Step 2:

[1615] The terminal sends the application information to the server. The terminal formats the input data and sends it to the server in the form of an HTTP request or similar. The server receives this request.

[1616] Input: Application information temporarily saved on your device

[1617] Output: Application information sent to the server

[1618] Step 3:

[1619] The server stores the received application information in a database. During this process, the server uses a database management system to check the integrity of the data and store the applicant information securely.

[1620] Input: Application information sent to the server

[1621] Output: Application information stored in a database

[1622] Step 4:

[1623] The store sets the pass / fail criteria through the management screen. For example, they can input criteria such as available working hours of "20 hours or more per week," related experience of "1 year or more," and communication skills of "high." The input criteria are sent from the terminal to the server.

[1624] Input: Pass / fail criteria entered by the store into the terminal

[1625] Output: Pass / fail criteria sent to the server

[1626] Step 5:

[1627] The server stores the received pass / fail criteria in a database, at which point the server checks the integrity of the data and adds the criteria to the database.

[1628] Input: Pass / fail criteria sent to the server

[1629] Output: Pass / fail criteria stored in the database

[1630] Step 6:

[1631] As the interview date and time approaches, the server configures the interview system based on the applicant's information and the criteria for acceptance or rejection, including the list of questions for the AI ​​interviewer and the interview schedule.

[1632] Input: Application information and acceptance criteria stored in the database

[1633] Output: Interview system settings

[1634] Step 7:

[1635] The user accesses the system through a terminal at the specified date and time and begins a conversation with the AI ​​interviewer. The terminal sends a connection request to the server, and the server launches the AI ​​interviewer.

[1636] Input: User access request

[1637] Output: Launched AI interviewer

[1638] Step 8:

[1639] The AI ​​interviewer asks questions to the applicant, and the user answers. This dialogue is sent to the server in real time. For example, the AI ​​interviewer asks, "How many hours can you work per week?" and the user replies, "I can work 20 hours per week."

[1640] Input: User's answer

[1641] Output: Real-time conversation logs recorded on the server

[1642] Step 9:

[1643] The emotion analyzer analyzes the user's tone of voice and facial expressions to determine their emotional state. For example, if the server detects that the user is nervous, it will dynamically change the AI ​​interviewer's questions to make them more relaxed.

[1644] Input: User's voice tone and facial expression data

[1645] Output: Emotional state analysis result

[1646] Step 10:

[1647] The server evaluates the user's responses based on the recorded dialogue logs and sentiment analysis results. This evaluation involves determining whether the user's available working hours and relevant experience meet the criteria, and using natural language processing to evaluate their communication skills.

[1648] Input: Dialogue logs and sentiment analysis results

[1649] Output: Evaluation results and overall score

[1650] Step 11:

[1651] After the pass / fail decision is complete, the server notifies the applicant of the result. For example, if the applicant passes, an email is sent to the user saying, "Congratulations! Here are the detailed steps to proceed to the next step." The user can then check the result on their device.

[1652] Input: Evaluation results and overall score

[1653] Output: Result email sent to applicant

[1654] (Application example 2)

[1655] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1656] Conventional recruitment interview systems did not take into account the emotional state of applicants during evaluation, resulting in problems with the fairness and impartiality of the interview. Furthermore, they were unable to analyze the emotional state of passengers in real time and optimize the in-car environment accordingly. This resulted in passengers' riding experiences not always being satisfactory. Therefore, the challenge is to provide a recruitment interview system that enables emotional analysis of applicants and provides fair and precise evaluations, as well as a system that provides a comfortable riding experience that reflects the emotional state of passengers.

[1657] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1658] In this invention, the server includes a terminal for inputting applicant information, a server that receives the applicant information transmitted from the terminal, a database connected to the server and having means for storing pass / fail criteria, and an interview system including an AI interviewer connected to the server and having means for engaging in a dialogue with the applicant, means for recording the dialogue and making an evaluation, means for analyzing the emotional state of passengers and dynamically adjusting the in-car environment, and means for making a pass / fail decision based on the evaluation and notifying the applicant of the result. This enables fair and precise evaluation that takes into account the emotional state of the applicant, and also enables the in-car environment to be optimized based on the emotional state of passengers, resulting in a more satisfying riding experience.

[1659] A "terminal" is a device through which applicants or customers enter information.

[1660] "Server" means a central computer group that receives and processes information sent from the Terminals.

[1661] "Database" means a storage device for storing pass / fail criteria and other relevant information.

[1662] An "AI interviewer" is software based on artificial intelligence that is used to interact with applicants.

[1663] The "interview system" is a set of mechanisms that includes an AI interviewer to interact with applicants, record the interaction, and evaluate it.

[1664] "Means for conducting evaluation" means a method or device for evaluating the abilities and qualifications of an applicant based on the dialogue records or other criteria.

[1665] The "means for determining whether an applicant passes or fails and notifying the applicant of the result" refers to a method or device for determining whether an applicant passes or fails based on the evaluation and notifying the applicant of the result.

[1666] The "means for analyzing emotional state" refers to a method or apparatus that analyzes voice tone and facial expressions to determine the user's emotions.

[1667] The "means for dynamically adjusting the in-car environment" refers to a method or device for automatically changing the background music, temperature, lighting, seat angle, etc. in the car based on the user's emotional state.

[1668] "Riding customers" are people riding in autonomous vehicles.

[1669] To implement this invention, a terminal for inputting applicant information is first prepared. The terminal can be a smartphone, tablet, or PC. The applicant uses the terminal to input information such as their name, available working hours, relevant experience, and self-promotion. The input information is sent to a server, which receives it and stores it in a database.

[1670] Next, the server receives the pass / fail criteria from the administrator and stores them in the database. The administrator sets the criteria such as available working hours, relevant experience, communication ability, etc. The pass / fail criteria are important indicators used to evaluate the job interview.

[1671] The server prepares an AI interviewer based on applicant information and pass / fail criteria. The AI ​​interviewer uses natural language processing technology to converse with the applicant. During the conversation, the server activates an emotion engine that analyzes the applicant's tone of voice and facial expressions to understand the applicant's emotional state. The emotion engine is built using software such as TensorFlow and OpenCV. For example, it can add questions to relax nervous applicants.

[1672] After the conversation is over, the server evaluates the applicant based on the content of the conversation and the results of sentiment analysis. Using natural language processing technology, the server evaluates the applicant's communication skills and calculates an overall score based on criteria such as available working hours and relevant experience. Based on this, the server determines whether the applicant is successful and notifies the applicant of the result.

[1673] The system also has the functionality to be used in autonomous vehicles. It can analyze the emotional state of passengers and dynamically adjust the in-car environment. Using the smartphone's camera and microphone, it analyzes the passenger's facial expressions and tone of voice and automatically adjusts the in-car background music, temperature, lighting, seat angle, and more, providing the optimal experience for passengers while they are in the vehicle.

[1674] Examples:

[1675] After a user gets into an autonomous vehicle, they launch the "Emotion Reader" app. The app uses the smartphone's camera and microphone to analyze the user's emotional state in real time. For example, if the app detects that the user's facial expression indicates "surprise" or "tension," it automatically adjusts the in-car background music to relaxing music. Furthermore, after a long ride, the app reports any changes in the user's emotions (from tension to relaxation).

[1676] Hardware and software used:

[1677] Smartphone: A device equipped with a camera and microphone to analyze the emotional state of passengers.

[1678] Python: The primary programming language used.

[1679] OpenCV: Used for real-time analysis of camera footage.

[1680] TensorFlow: Used to build the sentiment analysis model.

[1681] Firebase: Used for database and real-time updates.

[1682] Flask: A microframework for providing a sentiment analysis API on the backend.

[1683] Example prompt sentence:

[1684] "Tell me about a system in which a smartphone app inside a self-driving vehicle uses cameras and microphones to analyze passenger emotions in real time and optimize the ride experience (background music, temperature, lighting)."

[1685] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1686] Step 1:

[1687] Input: The applicant enters their name, available working hours, relevant experience, and self-promotional information into the terminal.

[1688] Output: The entered information is sent to the server.

[1689] Specific operation: The user enters the required information into the form on the device and presses the submit button. The device converts the entered information into JSON format and sends it to the server. The server saves the received information in a database in real time.

[1690] Step 2:

[1691] Input: The administrator inputs the pass / fail criteria from the terminal.

[1692] Output: The pass / fail criteria is sent to the server and stored in a database.

[1693] Specific operation: The administrator sets criteria such as available working hours, relevant experience, and communication skills on the management screen and sends them to the server, which receives them and stores them in the database.

[1694] Step 3:

[1695] Input: The server obtains applicant information and acceptance criteria.

[1696] Output: The AI ​​interviewer is prepared.

[1697] Specific operation: The server retrieves applicant information and pass / fail criteria from the database, and prepares an AI interviewer based on that information. The AI ​​interviewer loads a pre-set list of questions.

[1698] Step 4:

[1699] Input: The applicant uses the terminal to access the interview system.

[1700] Output: A conversation between the AI ​​interviewer and the applicant begins.

[1701] Specific operation: The applicant logs in to the system from their terminal at the designated interview date and time. The server authenticates the applicant and starts a dialogue session with the AI ​​interviewer. The dialogue is recorded in real time.

[1702] Step 5:

[1703] Input: AI interviewer analyzes applicant's tone of voice and facial expressions.

[1704] Output: Sentiment analysis results are obtained.

[1705] Specific operation: The server analyzes the camera footage using OpenCV to recognize the applicant's facial expressions, and analyzes the voice data using a TensorFlow model to analyze the voice tone. The analysis results are saved on the server.

[1706] Step 6:

[1707] Input: Dialogue records and sentiment analysis results are collected on the server.

[1708] Output: The applicant's overall rating is calculated.

[1709] How it works: The server uses natural language processing technology to evaluate the applicant's communication skills based on the dialogue log and sentiment analysis results, and calculates an overall score based on criteria such as available working hours and related experience.

[1710] Step 7:

[1711] Input: The server will determine whether the candidate passes or fails based on the overall evaluation.

[1712] Output: The applicant is notified of the results.

[1713] Specific operation: The server determines whether the applicant has passed or failed based on the overall evaluation score, and notifies the applicant of the result by email. The applicant receives the result and confirms the next steps.

[1714] Step 8:

[1715] Input: A customer enters an autonomous vehicle and launches a smartphone app.

[1716] Output: The in-car environment is dynamically adjusted based on the customer's emotional state.

[1717] How it works: The user launches the "Emotion Reader" smartphone app while in the car. The smartphone's camera and microphone are used to analyze the customer's facial expressions and tone of voice in real time. Based on the analysis results, the car's background music, temperature, lighting, seat angle, and other settings are automatically adjusted.

[1718] Example prompt sentence:

[1719] "Tell me about a system in which a smartphone app inside a self-driving vehicle uses cameras and microphones to analyze passenger emotions in real time and optimize the ride experience (background music, temperature, lighting)."

[1720] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1721] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1722] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1723] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1724] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1725] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1726] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1727] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1728] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1729] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1730] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1731] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1732] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1733] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1734] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1735] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1736] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1737] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1738] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1739] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1740] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1741] The following is further disclosed regarding the above embodiment.

[1742] (Claim 1)

[1743] a terminal for inputting applicant information;

[1744] a server that receives the applicant information transmitted from the terminal;

[1745] a database connected to said server, means for storing pass / fail criteria;

[1746] An interview system including an AI interviewer connected to the server, and means for engaging in dialogue with an applicant;

[1747] means for recording and evaluating said interactions;

[1748] a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result;

[1749] A system including:

[1750] (Claim 2)

[1751] 2. The system according to claim 1, further comprising means for evaluating the applicant based on a plurality of criteria such as the applicant's available working hours, related experience, and self-promotional content.

[1752] (Claim 3)

[1753] The system of claim 1 includes a means for analyzing questions posed by an AI interviewer and answers from applicants using natural language processing to evaluate communication skills.

[1754] "Example 1"

[1755] (Claim 1)

[1756] a terminal for inputting applicant information;

[1757] a server that receives the applicant information transmitted from the terminal and stores it in a database;

[1758] a database connected to the server, which stores pass / fail criteria;

[1759] An interview system including an AI interviewer connected to the server, a means for having a conversation with an applicant and recording the conversation content in real time;

[1760] means for analyzing the recorded dialogue log and performing evaluation using natural language processing;

[1761] a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result;

[1762] A system including:

[1763] (Claim 2)

[1764] 2. The system according to claim 1, further comprising means for evaluating the applicant based on a plurality of criteria such as the applicant's available working hours, related experience, and self-promotional content.

[1765] (Claim 3)

[1766] The system of claim 1 includes a means for analyzing questions posed by an AI interviewer and answers from applicants using natural language processing to evaluate communication skills.

[1767] "Application Example 1"

[1768] (Claim 1)

[1769] a terminal for inputting applicant information;

[1770] a server that receives the applicant information transmitted from the terminal;

[1771] a database connected to said server, means for storing pass / fail criteria;

[1772] An interview system including an AI interviewer connected to the server, and means for engaging in dialogue with an applicant;

[1773] A means for recording conversations between the AI ​​interviewer and the applicant and conducting real-time evaluations;

[1774] means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result via an application;

[1775] a means for notifying the applicant of the pass / fail result and providing information on the next procedure;

[1776] A system including:

[1777] (Claim 2)

[1778] The system of claim 1 includes a means for evaluating applicants based on multiple criteria, such as their available working hours, related experience, and self-promotional content, and calculating the results using a generative AI model.

[1779] (Claim 3)

[1780] The system of claim 1 further includes means for analyzing questions posed by an AI interviewer and answers from applicants using natural language processing to evaluate communication skills.

[1781] "Example 2: Combining Emotion Engines"

[1782] (Claim 1)

[1783] a terminal for inputting applicant information;

[1784] a server that receives the applicant information transmitted from the terminal;

[1785] a database connected to said server, means for storing pass / fail criteria;

[1786] an AI system connected to the server, which has means for interacting with applicants;

[1787] an emotion analysis device connected to the server, the emotion analysis device having means for analyzing an emotional state of a user;

[1788] means for recording and evaluating said interactions;

[1789] a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result;

[1790] A system including:

[1791] (Claim 2)

[1792] 2. The system according to claim 1, further comprising means for evaluating the applicant based on a plurality of criteria such as the applicant's available working hours, related experience, and self-promotional content.

[1793] (Claim 3)

[1794] The system of claim 1 includes a means for analyzing questions posed by an AI interviewer and answers from applicants using natural language processing to evaluate communication skills.

[1795] "Application example 2 when combining emotion engines"

[1796] (Claim 1)

[1797] a terminal for inputting applicant information;

[1798] a server that receives the applicant information transmitted from the terminal;

[1799] a database connected to said server, means for storing pass / fail criteria;

[1800] An interview system including an AI interviewer connected to the server, and means for engaging in dialogue with an applicant;

[1801] means for recording and evaluating said interactions;

[1802] A means for analyzing the emotional state of a passenger and dynamically adjusting the in-car environment;

[1803] a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result;

[1804] A system including:

[1805] (Claim 2)

[1806] 2. The system according to claim 1, further comprising means for evaluating the applicant based on a plurality of criteria such as the applicant's available working hours, related experience, and self-promotional content.

[1807] (Claim 3)

[1808] The system of claim 1 includes a means for analyzing questions posed by an AI interviewer and answers from applicants using natural language processing to evaluate communication skills. [Explanation of symbols]

[1809] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a terminal for inputting applicant information; a server that receives the applicant information transmitted from the terminal; a database connected to said server, means for storing pass / fail criteria; An interview system including an AI interviewer connected to the server, and means for engaging in dialogue with an applicant; means for recording and evaluating said interactions; a means for determining whether the applicant has passed or failed based on the evaluation and notifying the applicant of the result; A system including:

2. 2. The system according to claim 1, further comprising means for evaluating the applicant based on a plurality of criteria such as the applicant's available working hours, related experience, and self-promotional content.

3. The system according to claim 1, further comprising a means for analyzing questions posed by an AI interviewer and answers of an applicant using natural language processing to evaluate communication skills.

Images