System

Abstract

A system is provided.SOLUTION: A system for evaluating a candidate's skills, aptitude, and personality, comprising: means for collecting biometric data; means for the candidate to perform a skill test according to instructions displayed on a screen; means for analyzing the results of the skill test and the biometric data in real time and generating an overall score for the candidate; means for listing optimal candidates based on conditions preset by a company; and means for providing the optimal candidates to the company.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional hiring procedures rely solely on resumes and interviews to evaluate candidates, making it difficult to grasp their true skills and aptitude. Furthermore, the risk of inappropriate evaluation and hiring increases due to human bias. This is especially true when many candidates apply or when different skills are required, leading to the risk of overlooking talented individuals. The goal is to solve these problems and provide a fair and effective hiring process. [Means for solving the problem]

[0005] This invention provides a means for collecting biometric data and a means for candidates to take a skills test by following instructions displayed on the screen to objectively evaluate a candidate's skills, aptitude, and personality. It also provides a means for analyzing the collected skills test results and biometric data in real time to generate an overall score for the candidate. It configures a system that includes a means for creating a list of optimal candidates based on criteria preset by the company and providing them to the company. This eliminates traditional subjective evaluation methods, accurately assesses the true value of candidates, and realizes a fair hiring process.

[0006] "Biometric data" means data based on a candidate's biological information, including, for example, facial expressions, tone of voice, heart rate, eye movements, etc.

[0007] A "skills test" is a test administered to candidates to assess their skills, knowledge, and problem-solving abilities.

[0008] The "overall score" is a numerical evaluation of a candidate's skills, aptitude, and personality, calculated by combining the results of the skill test and biometric data.

[0009] "Real-time" refers to immediate response to ongoing processes or events without delay.

[0010] "Company conditions" are evaluation criteria such as the skill set, aptitude, and personality traits that the company is looking for.

[0011] "Listing" means creating a list of candidates selected based on specific criteria.

[0012] A "Candidate" is an individual who applies for an open position. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. In this system, various processes are performed based on the roles of the server, terminal, and user.

[0035] Server Operation

[0036] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by companies in a database. This allows candidates to be evaluated based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate. This creates a foundation for evaluating candidates in a standardized manner.

[0037] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the device to evaluate the candidate's stress level and concentration. Based on this, the server combines the skill assessment and biometric assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0038] Device behavior

[0039] The terminal is the device used by the candidate and provides the interface for performing the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. The terminal also uses cameras, microphones, and sensors to collect biometric data and transmits the data to the server. This data includes the candidate's facial expressions, tone of voice, heart rate, etc.

[0040] User Actions

[0041] The user is a candidate and creates an account by entering the necessary personal information on the platform when first accessing it. After logging in, the user takes a skill test provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user checks the results and understands their strengths and areas for improvement.

[0042] Specific examples

[0043] For example, if a company sets "programming ability" and "stress management ability" as requirements, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers, as well as facial expressions and tone of voice, and sends them to the server.

[0044] The server scores Candidate A's programming skills and evaluates his / her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's desired criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's criteria.

[0045] In this way, the present invention eliminates bias and ensures an objective and fair hiring process.

[0046] The processing flow will be explained below.

[0047] Step 1:

[0048] server

[0049] The desired skill sets, aptitude conditions, and evaluation criteria provided by companies are stored in a database, which allows evaluation criteria to be set based on each company's unique conditions.

[0050] Step 2:

[0051] Terminal

[0052] Activate and verify that your cameras, microphones, and sensors are working properly, so they are ready to accurately collect biometric data.

[0053] Step 3:

[0054] User

[0055] Access the platform, enter the required personal information to create an account, and after logging in, access the skill test interface provided by the server.

[0056] Step 4:

[0057] server

[0058] The system verifies the user's login information and displays the appropriate skills test on the screen. The user is then ready to begin the skills test.

[0059] Step 5:

[0060] Terminal

[0061] As the user starts the skill test, the system provides an input interface and transmits the user's input data to the server in real time, and also collects the candidate's biometric data and transmits it to the server.

[0062] Step 6:

[0063] User

[0064] Follow the on-screen instructions to complete the skills test, answering each task in turn and providing additional information as needed.

[0065] Step 7:

[0066] server

[0067] The system analyzes the skill test response data received in real time to quantify the candidate's skill level, while analyzing the biometric data sent from the device to evaluate the candidate's stress level and concentration.

[0068] Step 8:

[0069] server

[0070] Skill assessment and biometric assessment are combined to generate an overall score for the candidate, which reflects the candidate's skills, aptitudes, and personality.

[0071] Step 9:

[0072] server

[0073] Based on the generated overall score, the system compares the results with the company's requirements to create a list of the most suitable candidates, and provides information about the candidates to the company.

[0074] Step 10:

[0075] Terminal

[0076] It displays a detailed report of completed skill tests and assessment results to users, allowing them to receive feedback on their skills and aptitudes.

[0077] Step 11:

[0078] company

[0079] The company reviews the list of candidates provided by the server and plans interviews or additional evaluations as the next step. Finally, the company hires the best candidate.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Traditional hiring processes are prone to subjective evaluation and bias, making it difficult to conduct fair evaluations. It's also difficult to accurately assess applicants' skills and aptitude, making it difficult to hire the best candidates. Furthermore, important factors like a candidate's stress level and level of concentration tend to be overlooked, resulting in cases where candidates don't accurately match the talent specifications companies are looking for.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes a means for storing the skill sets, aptitude criteria, and evaluation criteria provided by the company in a database, a means for analyzing the skill test results and biometric data in real time to generate an overall score for the candidate, and a means for integrating the skill evaluation and biometric evaluation to generate an overall score. This eliminates subjective bias and enables the evaluation of candidates in an objective and fair manner. Factors such as stress level and concentration are also taken into account, making it possible to list the most suitable candidates who meet the company's desired criteria.

[0085] "Biometric data" refers to data that measures a candidate's physical characteristics and is stored digitally, such as facial expressions, tone of voice, and heart rate.

[0086] A "skills test" is a test administered to assess a candidate's specific abilities or knowledge. Examples include programming tests and tasks that assess problem-solving ability.

[0087] The "total score" is an evaluation of a candidate calculated based on the results of the skill test and biometric data. It is used to comprehensively judge the candidate's abilities and aptitude.

[0088] The "database" is an information management system for storing the skill sets, aptitude conditions, and evaluation criteria provided by the company. Specifically, a database management system such as MySQL is used.

[0089] "Real-time analysis" is the process of analyzing skill test results and biometric data as they are collected, allowing for rapid candidate evaluation.

[0090] "A means of listing the best candidates based on a company's requirements" is a method for selecting candidates with a high overall score based on evaluation criteria set by the company in advance. This method allows companies to efficiently identify the talent they are looking for.

[0091] "Cameras, microphones, and sensors" are hardware devices used to collect biometric data on candidates, including facial expressions, tone of voice, and heart rate.

[0092] "Means for quantifying skill levels" refers to a method for expressing a candidate's specific abilities as a number based on the results of a skill test. This allows for an objective evaluation of the candidate's abilities.

[0093] This invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. This system performs various processes based on the roles of the server, terminal, and user.

[0094] Server Operation

[0095] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by companies in a database. This allows candidates to be evaluated based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate. This creates a foundation for evaluating candidates in a standardized manner.

[0096] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the device to evaluate the candidate's stress level and concentration. Based on this, the server combines the skill assessment and biometric assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0097] Specific hardware and software features include a server running Python and a MySQL database for data processing and analysis, and TensorFlow and Keras machine learning algorithms for analyzing biometric data.

[0098] Device behavior

[0099] The terminal is the device used by the candidate and provides the interface for performing the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. The terminal also uses cameras, microphones, and sensors to collect biometric data and transmits the data to the server. This data includes the candidate's facial expressions, tone of voice, heart rate, etc.

[0100] Specific hardware such as cameras, microphones, and sensors are required to accurately collect biometric data, which can be used to assess a candidate's stress level, concentration, etc. in real time.

[0101] User Actions

[0102] The user is a candidate and creates an account by entering the necessary personal information on the platform when first accessing it. After logging in, the user takes a skill test provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user checks the results and understands their strengths and areas for improvement.

[0103] Specific examples

[0104] For example, if a company sets "programming ability" and "stress management ability" as requirements, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers, as well as facial expressions and tone of voice, and sends them to the server.

[0105] The server scores Candidate A's programming skills and evaluates his / her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's desired criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's criteria.

[0106] Prompt Sentence Examples

[0107] An example prompt is:

[0108] "What methods and techniques will the system use to assess candidates' programming and stress management abilities?"

[0109] In this way, the present invention eliminates bias and ensures an objective and fair hiring process.

[0110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0111] Step 1: Register your company information

[0112] The server stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. The input is information provided by the company (e.g., programming ability, teamwork ability). The server stores this information in the database, and the company's conditions are registered in the database as output.

[0113] Step 2: Register for the skills test

[0114] The server registers the skill tests and assignments provided by the company in a database. The input is the skill tests and assignments (e.g. Java programming tests) provided by the company. The server saves these in the database, and the registered skill tests and assignments are stored in the database as output.

[0115] Step 3: Create a candidate account

[0116] A user creates an account by entering the necessary personal information when first accessing the site. The input includes the personal information provided by the user (e.g., name, email address, password). The server stores this information in a database, and the output is a user account.

[0117] Step 4: Start the skills test

[0118] After logging in, the user starts the skill test provided by the server. The input is the user's login information and a command to start the test. The server starts the skill test based on this, and the test screen is displayed on the terminal as output.

[0119] Step 5: Perform a skills test

[0120] The user takes the skill test by following the instructions displayed on the screen. The input is each question in the test that is displayed on the screen. The user answers the questions, and the device sends the answers to the server in real time. The output is the answer data sent to the server.

[0121] Step 6: Collecting Biometric Data

[0122] The terminal collects biometric data using cameras, microphones, and sensors. The inputs include data such as the candidate's facial expression, tone of voice, and heart rate. The terminal collects this data and sends it to the server. The output is the biometric data sent to the server.

[0123] Step 7: Analyze the data

[0124] The server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data to evaluate the candidate's stress level and concentration. The inputs are the response data and biometric data. The server analyzes the data using Python scripts and TensorFlow, and the output is the evaluation results.

[0125] Step 8: Generate an overall score

[0126] The server combines the skill assessment and biometric assessment to generate an overall score. The inputs are the results of the skill assessment and biometric assessment. The server combines these assessments and calculates an overall score as output.

[0127] Step 9: Shortlist the best candidates

[0128] The server compares the overall score with the company's requirements and lists the most suitable candidates. The input is the overall score and the company's requirements. The server compares these and generates a list of the most suitable candidates as output.

[0129] Step 10: Providing the best candidates

[0130] The server provides companies with the best candidates. As input, it has the information of the candidates listed. The server provides this to the company and as output, it provides the information in a format that the company can view.

[0131] This process allows the system to eliminate bias and ensure an objective and fair hiring process.

[0132] (Application example 1)

[0133] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0134] Traditional recruitment systems can be biased when evaluating candidates' skills, aptitude, and personality, making it difficult to evaluate them fairly. Furthermore, systems that evaluate the effectiveness of advertising are unable to incorporate real-time responses from viewers, making it difficult to provide personalized advertising. For these reasons, there is a need for more accurate and objective evaluation systems in both the fields of candidate evaluation and ad personalization.

[0135] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0136] In this invention, the server includes means for collecting biometric data to evaluate the skills, aptitude, and personality of candidates, means for the candidate to take a skill test according to instructions displayed on the screen, means for analyzing the skill test results and biometric data in real time to generate an overall score for the candidate, means for compiling a list of optimal candidates based on conditions preset by the company, means for providing the optimal candidates to the company, means for collecting biometric data of advertisement viewers and personalizing advertisements, and means for measuring the effectiveness of advertisements in real time and providing feedback on the results, thereby enabling fair and efficient candidate evaluation and highly accurate advertisement personalization.

[0137] A "candidate" is an individual with specific skills and aptitudes who applies for a role or position sought by a company.

[0138] "Biometric data" is data that digitally records an individual's physiological or behavioral characteristics, such as heart rate, facial expression, and tone of voice.

[0139] A "skills test" is an exam or task designed to assess a candidate's specific knowledge or abilities.

[0140] "Means of real-time analysis" refers to technologies and devices that process data immediately and reflect the results instantly.

[0141] "Composite score" is a number or indicator that indicates the candidate's overall evaluation, generated by integrating the results of the skill test and biometric data.

[0142] "Conditions set in advance by the company" are the standards set in advance by the company regarding the skills, aptitudes, and other requirements that the company requires of job candidates.

[0143] A "means of shortlisting the best candidates" is a method or technique for selecting the most suitable candidates based on set criteria.

[0144] An "advertising viewer" is an individual who views an advertisement and is the person to whom the advertisement is directed.

[0145] "Means for personalizing advertising" are methods and technologies for individually tailoring advertising content based on the viewer's characteristics, interests, and responses.

[0146] "Means for measuring the effectiveness of advertising in real time and providing feedback on the results" refers to a system that instantly analyzes the reactions of viewers of advertisements and reflects the results in advertising strategies.

[0147] This invention is a system that objectively evaluates candidates' skills, aptitude, and personality to realize a fair hiring process. The system includes a server, a terminal, and a user. Furthermore, the system can collect responses from advertisement viewers, personalize advertisements, and measure their effectiveness.

[0148] Server Operation

[0149] The server is responsible for storing the desired skills, aptitudes, and evaluation criteria provided by companies in a database. Based on the conditions set by the company, the server manages the candidates' skill tests and assignments and generates an evaluation profile for each candidate.

[0150] When a candidate begins a skill test, the server receives and analyzes the skill test response data and biometric data in real time. This analysis is primarily performed using software such as OpenCV, TensorFlow, and Keras. This integrates the candidate's skill assessment and biometric assessment to generate an overall score. In addition, the server also collects biometric data of ad viewers, measures the effectiveness of personalized advertising, and provides feedback on the results.

[0151] Device behavior

[0152] The terminal is a device used by candidates and ad viewers, and provides an interface for taking skill tests and viewing ads. The terminal is equipped with a camera, microphone, and various sensors, and these devices are used to collect biometric data.

[0153] During the skill test, the device transmits the candidate's input in real time to a server. The device also collects biometric information such as facial expressions, tone of voice, and heart rate of the ad viewer and transmits this information to the server, which allows for personalized advertising.

[0154] User Actions

[0155] Users are candidates or ad viewers, and when they first access the platform, they create an account by entering the necessary personal information. After logging in, candidates take a skill test provided by the server. During the skill test, users answer questions in order by following the instructions displayed on the screen. Once the skill test is complete, users can check the results and identify their strengths and areas for improvement.

[0156] Ad viewers' responses are collected in real time as they watch the ad, and based on their responses, ad content is personalized and reflected in future viewings.

[0157] Specific examples

[0158] For example, if a company sets "programming ability" and "stress management ability" as required criteria, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers as well as facial expressions and tone of voice, and sends these to the server. The server then scores Candidate A's programming skills and evaluates his or her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's required criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0159] In addition, when an ad viewer is viewing a specific ad, their reaction can be collected and analyzed in real time and used to determine the next ad display. A specific example of a prompt sentence could be entered as follows:

[0160] "We want to develop a system that analyzes users' emotional responses and stress levels in real time, and then delivers personalized advertising content that is optimal for them. We will collect and analyze biometric data such as facial expressions, heart rate, and tone of voice, and use this data to evaluate the effectiveness of advertising. As part of ad personalization, we want to reflect content that the viewer is particularly interested in when displaying ads next time."

[0161] In this way, the present invention improves the fairness of candidate evaluation and enables advertising personalization and effectiveness measurement.

[0162] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0163] Step 1:

[0164] The server stores the desired skill sets, aptitudes, and evaluation criteria provided by companies in a database. This creates a foundation for evaluating candidates based on each company's unique criteria. The input is the requirements data provided by the company, and the output is the evaluation criteria stored in the database.

[0165] Step 2:

[0166] The server registers various skill tests and assignments in a database and generates an individual assessment profile for each candidate. The input is the test question data, and the output is the assessment profile for each candidate. This data is used for subsequent analysis.

[0167] Step 3:

[0168] The candidate starts the skill test using the terminal. The terminal sends the candidate's input to the server in real time. The input is the candidate's answer data, and the output is the server that received the data. The server prepares to analyze the received data immediately.

[0169] Step 4:

[0170] The device uses cameras, microphones, and sensors to collect biometric data (facial expressions, tone of voice, heart rate, etc.) of the candidate and transmits it to a server. The input is the collected biometric data, and the output is the data transmitted to the server.

[0171] Step 5:

[0172] The server analyzes the skill test response data and biometric data in real time. Software such as OpenCV, TensorFlow, and Keras is used for the analysis. The input is the response data and biometric data, and the output is the analysis results. Specific operations include facial expression recognition using models, voice tone analysis, and heart rate variability analysis.

[0173] Step 6:

[0174] The server generates an overall score for the candidate based on the analysis results. The input is the analyzed data, and the output is the overall score, which integrates the candidate's skills and stress assessment.

[0175] Step 7:

[0176] The server compares the overall score with the company's set conditions and lists the most suitable candidates. The input is the overall score and the company's conditions, and the output is a list of the most suitable candidates.

[0177] Step 8:

[0178] The server provides companies with the best candidates. The input is the candidate information listed, and the output is the candidate information sent to the company.

[0179] Step 9:

[0180] The ad viewer watches the ad using a device. The device uses a camera, microphone, and sensors to collect the viewer's responses and send them to a server. The input is biometric data from the time the ad was viewed, and the output is the biometric data sent to the server.

[0181] Step 10:

[0182] The server analyzes the biometric data of the ad viewers in real time to measure the effectiveness of the ad. The input is the viewer's biometric data, and the output is the analysis result of the ad's effectiveness. This analysis includes evaluating emotional responses and measuring stress levels.

[0183] Step 11:

[0184] The server personalizes the ads based on the analysis results and provides the most suitable ads to the viewer. The input is the analysis result of the advertising effectiveness, and the output is personalized advertising content. This result is reflected in subsequent viewings.

[0185] Examples:

[0186] For example, if a viewer shows strong interest in a particular ad, the next ad display can be optimized based on that reaction data. This is achieved using technologies such as "generative AI models and prompt sentences."

[0187] Example prompt sentence:

[0188] "We want to develop a system that analyzes users' emotional responses and stress levels in real time, and then delivers personalized advertising content that is optimal for them. We will collect and analyze biometric data such as facial expressions, heart rate, and tone of voice, and use this data to evaluate the effectiveness of advertising. As part of ad personalization, we want to reflect content that the viewer is particularly interested in when displaying ads next time."

[0189] In this way, a system is built that achieves high accuracy and fairness in both candidate evaluation and ad personalization.

[0190] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0191] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. This system includes a means for incorporating an emotion engine to recognize candidates' emotions from biometric data and reflect them in the evaluation.

[0192] Server Operation

[0193] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. This allows evaluation criteria to be set based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate.

[0194] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the terminal and recognizes the candidate's emotions using an emotion engine. Based on this, the server integrates the skill assessment, biometric assessment, and emotion assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0195] Device behavior

[0196] The terminal is the device used by the candidate and provides the interface for taking the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. At the same time, the terminal uses a camera, microphone, and sensors to collect biometric data, which is then transmitted to the server. This biometric data includes facial expressions, tone of voice, heart rate, etc.

[0197] User Actions

[0198] The user is a candidate and creates an account by entering the required personal information on the platform when first accessing it. After logging in, the user accesses the skill test interface provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user can check the results and understand their strengths and areas for improvement.

[0199] Specific examples

[0200] For example, if a company sets the requirements for a candidate as "programming ability," "stress management ability," and "emotional stability," the system works as follows:

[0201] Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers as well as facial expressions and tone of voice, and sends these to the server. The emotion engine analyzes the collected data, recognizes Candidate A's emotions in real time, and evaluates their stress level and concentration.

[0202] The server scores Candidate A's programming skills and generates an overall score by combining the emotional evaluation from the emotion engine and the biometric evaluation. Based on this overall score, the server compares the candidates with the company's requirements and creates a list of the most suitable candidates. The company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0203] In this way, the present invention eliminates bias and provides an objective and fair hiring process, allowing companies to effectively select top talent.

[0204] The processing flow will be explained below.

[0205] Step 1:

[0206] server

[0207] The desired skill sets, aptitude conditions, and evaluation criteria provided by companies are stored in a database, which allows evaluation criteria to be set based on each company's unique conditions.

[0208] Step 2:

[0209] Terminal

[0210] Activate and verify that cameras, microphones, and sensors are working properly, so they are ready to accurately collect biometric and emotional data.

[0211] Step 3:

[0212] User

[0213] Access the platform, enter the required personal information to create an account, and after logging in, access the skill test interface provided by the server.

[0214] Step 4:

[0215] server

[0216] The system verifies the user's login information and displays the appropriate skills test on the screen. The user is then ready to begin the skills test.

[0217] Step 5:

[0218] Terminal

[0219] As the user starts the skill test, the system provides an input interface and transmits the user's input data to the server in real time. It also collects biometric data using cameras, microphones, and sensors and transmits this data to the server.

[0220] Step 6:

[0221] User

[0222] Follow the on-screen instructions to complete the skills test, answering each task in turn and providing additional information as needed.

[0223] Step 7:

[0224] server

[0225] The system analyzes the skill test response data received in real time and quantifies the candidate's skill level. At the same time, it uses an emotion engine to analyze the biometric data sent from the device and recognize the candidate's emotions. Based on the recognized emotional data, it evaluates the candidate's stress level and concentration level.

[0226] Step 8:

[0227] server

[0228] Skill assessment, emotional assessment, and biometric assessment are combined to generate an overall score for the candidate, which reflects the candidate's skills, aptitudes, emotions, and personality.

[0229] Step 9:

[0230] server

[0231] Based on the generated overall score, the system compares the results with the company's requirements to create a list of the most suitable candidates, and provides information about the candidates to the company.

[0232] Step 10:

[0233] Terminal

[0234] It displays a detailed report of completed skill tests and assessment results to users, allowing them to receive feedback on their skills and aptitudes.

[0235] Step 11:

[0236] company

[0237] The company reviews the list of candidates provided by the server and plans interviews or additional evaluations as the next step. Finally, the company hires the best candidate.

[0238] Example 2

[0239] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0240] In the traditional recruitment process, it is difficult to objectively evaluate a candidate's skills, aptitude, and personality, and the evaluator's subjectivity and bias are easily incorporated, making it difficult to conduct a fair evaluation and select the right talent.In addition, because the evaluation does not take into account the candidate's emotions or stress levels, it is difficult to accurately predict how a candidate will perform in a real work environment.

[0241] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0242] In this invention, the server includes means for storing the skill sets, aptitude conditions, and evaluation criteria provided by the company in a database, means for registering various skill tests and tasks in the database and generating an individual evaluation profile for each candidate, means for receiving and analyzing candidate skill test response data in real time, means for analyzing biometric data and recognizing the candidate's emotions using an emotion engine, means for generating an overall score by integrating the skill evaluation, biometric evaluation, and emotion evaluation, means for comparing the overall score with the company's conditions to list the most suitable candidates, and means for providing the most suitable candidates to the company. This makes it possible to objectively evaluate the skills, aptitude, and personality of candidates and realize a fair hiring process that eliminates bias.

[0243] "Company conditions" refers to the skill sets, aptitudes, and evaluation criteria required by the company.

[0244] "Database" refers to a system for storing and managing data such as information provided by companies, skill tests, assignments, and candidate evaluation profiles.

[0245] A "skills test" refers to a series of questions or tasks administered to assess a candidate's abilities or skills.

[0246] "Biometric data" refers to a candidate's biological information, specifically facial expressions, tone of voice, heart rate, etc.

[0247] An "emotion engine" refers to software or algorithms that analyze collected biometric data and recognize a candidate's emotions.

[0248] "Skills assessment" refers to a method of quantifying the results of a candidate's skills test and evaluating their abilities.

[0249] "Biometric assessment" refers to a method of assessing a candidate's status or characteristics based on collected biometric data.

[0250] "Emotional assessment" refers to an assessment based on the candidate's emotional state recognized using an emotion engine.

[0251] "Composite Score" refers to a candidate's overall assessment score generated by integrating skill assessment, biometric assessment, and emotional assessment.

[0252] The "best candidate" is the candidate who best meets the company's requirements.

[0253] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. The system includes a means for incorporating candidates' emotions into the evaluation by combining an emotion engine with analysis of biometric data.

[0254] Server Operation

[0255] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. Specifically, by registering the information provided by the company in the database, each company's unique criteria are saved. The server also registers skill tests and assignments in the database. Processing is then carried out to generate an individual evaluation profile for each candidate.

[0256] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, the server receives biometric data sent from the device. This biometric data includes facial expressions, tone of voice, heart rate, etc. The server uses an emotion engine to analyze the biometric data and recognize the candidate's emotions.

[0257] The server combines the skill, biometric, and emotional assessments to generate a comprehensive score, which is then matched with the company's requirements to create a list of the most suitable candidates and provide them to the company.

[0258] Device behavior

[0259] The terminal is the device used by the candidate and provides the interface for the skills test. When the candidate starts the skills test, the terminal transmits answer data in real time to the server, including all data entered by the candidate.

[0260] At the same time, the device collects biometric data using cameras, microphones, and sensors, such as a webcam, microphone, and heart rate sensor, and transmits this biometric data to a server to assess the candidate's emotional state.

[0261] User Actions

[0262] The user is a candidate, and when accessing the platform for the first time, they create an account by entering the necessary personal information. After logging in, they access the skill test interface provided by the server. The candidate answers the questions by following the instructions displayed on the screen.

[0263] Once the skills test is complete, users can review their results to identify their strengths and areas for improvement.

[0264] Specific examples

[0265] For example, if a company sets requirements for "programming skills," "stress management skills," and "emotional stability," the system works as follows:

[0266] Candidate A logs into the platform and takes a programming skills test.

[0267] During the test, the device collects Candidate A's answers as well as his / her facial expressions and tone of voice, and sends them to the server.

[0268] The emotion engine analyzes the collected data, recognizes Candidate A's emotions in real time, and assesses his stress level and concentration.

[0269] The server scores Candidate A's programming skills, and generates an overall score by combining the emotional evaluation by the emotion engine and the biometric evaluation. Based on this overall score, the server compares the candidates with the company's requirements and creates a list of the most suitable candidates.

[0270] The company checks the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0271] In this way, the present invention eliminates bias and provides an objective and fair hiring process, allowing companies to effectively select top talent.

[0272] Example prompts to input to the generative AI model

[0273] System Overview: A system that objectively evaluates candidates' skills, aptitude, and personality to ensure a fair hiring process.

[0274] Technologies used: Emotion engine, biometric data analysis

[0275] Example: A company sets evaluation criteria such as "programming ability," "stress management ability," and "emotional stability."

[0276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0277] Step 1: Store company information in a database

[0278] The server receives the skill sets, aptitude conditions, and evaluation criteria provided by the companies and stores them in a database.

[0279] Input: Skill set, qualifications, and evaluation criteria provided by the company

[0280] Output: Company condition information stored in the database

[0281] Specific operation: The server acquires skill sets such as "Java programming," "project management," and "communication skills" sent by Company A and stores them in a database called "Company A_Evaluation Criteria."

[0282] Step 2: Register for the Skills Test and Assignment

[0283] The server registers various skill tests and assignments in a database and generates an evaluation profile for each candidate.

[0284] Input: Skill test, assignment information

[0285] Output: Assessment profile for each candidate

[0286] Specific operation: The server registers the "Java Programming Proficiency Test" and "Project Presentation Assignment" in the database and creates a "Candidate A_Evaluation Profile" for Candidate A.

[0287] Step 3: Initiating the skill test and receiving response data

[0288] When a candidate starts a skill test, the device sends answer data to the server in real time.

[0289] Input: Candidate's answer data

[0290] Output: Answer data sent to the server

[0291] Specific operation: Every time the user clicks "Next Question," the device sends the answer data to the server in real time.

[0292] Step 4: Collect and analyze biometric data

[0293] The device uses a camera, microphone, and sensors to collect biometric data and transmits it to a server, which then uses an emotion engine to analyze the biometric data and recognize the candidate's emotions.

[0294] Input: Biometric data (facial expression, tone of voice, heart rate)

[0295] Output: Parsed emotion data

[0296] Specific operation: The device captures facial expressions with a camera, records voice tone with a microphone, and measures heart rate with a heart rate sensor, and sends this data to a server. The emotion engine analyzes the received data and calculates stress levels and concentration levels.

[0297] Step 5: Integrate the data and generate an overall score

[0298] The server combines the skill assessment, biometric assessment, and emotional assessment to generate an overall score.

[0299] Input: Skill assessment data, Biometric assessment data, Emotion assessment data

[0300] Output: Overall score

[0301] Specific operation: The server calculates a skill score (e.g., "85 points / 100 points") based on the answer data, and also performs emotional evaluation using an emotion engine and biometric evaluation, integrating these to generate "Candidate A_Overall Score" (e.g., "Score: 90 / 100").

[0302] Step 6: Matching with companies and providing candidate lists

[0303] The server compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0304] Input: Overall score, company conditions

[0305] Output: A list of the best candidates

[0306] Specific operation: The server compares "Candidate A_Total Score" with the conditions of Company A, determines from the comparison results that "Candidate A meets the conditions of Company A" and adds him to the list. The company then checks this list and selects the most suitable candidate.

[0307] (Application example 2)

[0308] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0309] In the modern hiring process, it is difficult to fairly evaluate a candidate's skills, aptitude, and personality. Furthermore, these evaluations tend to rely on subjective judgment and are prone to bias. For candidates working in retail stores, evaluating their customer service attitude and stress tolerance is particularly important, but traditional methods make these evaluations difficult. Furthermore, there is a lack of technology that can grasp a candidate's emotional state in real time and reflect that information in the overall evaluation.

[0310] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting biometric data to evaluate the skills, aptitude, and personality of candidates; means for the candidate to take a skill test according to instructions displayed on the screen; means for analyzing the skill test results and biometric data in real time and generating an overall score for the candidate; means for comparing the results with registered evaluation criteria and listing the candidates who best meet the conditions; means for providing the most appropriate candidates to the company; means for analyzing facial expressions and voice in real time using a camera, microphone, and sensor to evaluate the candidate's emotions and stress level; and means for presenting the evaluation results using an overall score based on the customer service attitude and stress tolerance set by the company. This enables a comprehensive evaluation of the candidate's skills and emotional state and a fair and comprehensive judgment.

[0311] A "candidate" is a person who is being evaluated during the recruitment process.

[0312] "Skills" are specific skills or abilities that a candidate possesses.

[0313] "Aptitude" refers to a candidate's ability to fit into a particular job or environment.

[0314] "Personality" refers to the candidate's character and behavioral traits.

[0315] "Biometric data" refers to data based on a candidate's biological information, such as facial expression analysis, voice analysis, and heart rate.

[0316] A "skills test" is an examination to evaluate a candidate's skills and abilities.

[0317] The "total score" is an evaluation index calculated by integrating the results of the skill test and biometric data.

[0318] The "emotion recognition model" is an artificial intelligence model that recognizes candidates' emotions in real time from biometric data.

[0319] "Real-time analysis" is the process of processing data in real time and outputting the results immediately.

[0320] "Evaluation criteria" are standard values ​​used to quantify the skills and aptitudes that companies are looking for.

[0321] "Emotional state" refers to the candidate's current emotional state, including emotions such as joy, anger, sadness, and happiness.

[0322] "Stress tolerance" refers to a candidate's ability to resist and adapt to stress.

[0323] "Customer service attitude" refers to the attitude and behavior displayed by the candidate when performing customer service duties.

[0324] "Company" refers to the organization or entity seeking to recruit a Candidate.

[0325] The present invention provides a system for objectively evaluating a candidate's skills, aptitude, and personality to realize a fair hiring process. Specific embodiments are described below.

[0326] Server Operation

[0327] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. This allows evaluation criteria to be set based on each company's unique conditions. Next, the server registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate.

[0328] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it processes the biometric data sent from the device and recognizes the candidate's emotions using an emotion recognition model. Based on this, the server integrates the skill assessment, biometric assessment, and emotion assessment to generate an overall score. Finally, it compares this overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0329] Device behavior

[0330] The terminal is the device used by the candidate and provides the interface for taking the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. At the same time, the terminal uses a camera, microphone, and sensors to collect biometric data, which is then transmitted to the server. This biometric data includes facial expressions, tone of voice, heart rate, etc.

[0331] User Actions

[0332] The user is a candidate and creates an account by entering the required personal information on the platform when first accessing it. After logging in, the user accesses the skill test interface provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user can check the results and understand their strengths and areas for improvement.

[0333] Hardware and software used

[0334] The hardware used in implementing this invention includes a smartphone, smart glasses, and a webcam. The software used includes OpenCV, Dlib, Keras (with TensorFlow backend), Python, and emotion recognition models. This combination of hardware and software allows for real-time data collection, analysis, and display.

[0335] Specific examples

[0336] For example, if a candidate is considering a position at a high-end restaurant in Tokyo, the system will evaluate the candidate's smile, nervousness, and friendliness in real time during the interview. In this example, the following prompts are used:

[0337] "Among your friends and family, what are the characteristics of people you find particularly approachable? Please describe specifically what their facial expressions are like."

[0338] The prompts allow candidates to be assessed on their emotional stability and customer service attitude, resulting in an overall score, enabling companies to ensure an objective and fair hiring process and effectively select the best candidates.

[0339] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0340] Step 1:

[0341] When a user first accesses the platform, they create an account by entering the necessary personal information, including basic information such as name, contact details, and resume, and the server receives this data and stores it in a database.

[0342] Step 2:

[0343] The user logs in and accesses the skill test interface provided by the server. Based on the login information, the server generates a skill test interface dedicated to the user and displays it on the screen.

[0344] Step 3:

[0345] The user follows the on-screen instructions to begin the skill test, and each response is sent in real time from the device to the server, which analyzes the response data and monitors the user's progress.

[0346] Step 4:

[0347] The device will begin collecting biometric data about the user using cameras, microphones, and sensors, including facial expressions, tone of voice, and heart rate, in real time, and will send this data to a server.

[0348] Step 5:

[0349] The server inputs the received biometric data into an emotion recognition model to analyze the user's emotional state. The emotion recognition model uses facial expression recognition and voice analysis algorithms to assess the user's current emotions and stress levels.

[0350] Step 6:

[0351] The server combines the skill test results with the emotional assessment obtained from biometric data to generate an overall score. Skill assessment is based on the accuracy rate and speed of answers, while emotional assessment is based on emotional stability and stress tolerance.

[0352] Step 7:

[0353] The server compares the overall score with the evaluation criteria set by the company and lists the candidates who best fit the criteria. This determines how well the candidate's overall evaluation matches the company's requirements.

[0354] Step 8:

[0355] The server provides companies with the best candidates and allows them to view the assessment results on a dashboard, including detailed skill test results, biometric assessment, and overall score.

[0356] Step 9:

[0357] Based on the results of the assessment, companies can decide on interviews or next steps in the selection process. Using the dashboard, company personnel can view detailed assessments of candidates and input further questions and prompts into the generative AI model for further assessment.

[0358] For example, the following prompt text can be used:

[0359] "Among your friends and family, what are the characteristics of people you find particularly approachable? Please describe specifically what their facial expressions are like."

[0360] This process allows companies to comprehensively evaluate a candidate's skills and emotional state, allowing them to make a fair and comprehensive decision.

[0361] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0362] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0363] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0364] [Second embodiment]

[0365] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0366] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0367] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0368] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0369] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0370] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0371] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0372] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0373] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0374] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0375] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0376] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0377] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. In this system, various processes are performed based on the roles of the server, terminal, and user.

[0378] Server Operation

[0379] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by companies in a database. This allows candidates to be evaluated based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate. This creates a foundation for evaluating candidates in a standardized manner.

[0380] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the device to evaluate the candidate's stress level and concentration. Based on this, the server combines the skill assessment and biometric assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0381] Device behavior

[0382] The terminal is the device used by the candidate and provides the interface for performing the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. The terminal also uses cameras, microphones, and sensors to collect biometric data and transmits the data to the server. This data includes the candidate's facial expressions, tone of voice, heart rate, etc.

[0383] User Actions

[0384] The user is a candidate and creates an account by entering the necessary personal information on the platform when first accessing it. After logging in, the user takes a skill test provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user checks the results and understands their strengths and areas for improvement.

[0385] Specific examples

[0386] For example, if a company sets "programming ability" and "stress management ability" as requirements, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers, as well as facial expressions and tone of voice, and sends them to the server.

[0387] The server scores Candidate A's programming skills and evaluates his / her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's desired criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's criteria.

[0388] In this way, the present invention eliminates bias and ensures an objective and fair hiring process.

[0389] The processing flow will be explained below.

[0390] Step 1:

[0391] server

[0392] The desired skill sets, aptitude conditions, and evaluation criteria provided by companies are stored in a database, which allows evaluation criteria to be set based on each company's unique conditions.

[0393] Step 2:

[0394] Terminal

[0395] Activate and verify that your cameras, microphones, and sensors are working properly, so they are ready to accurately collect biometric data.

[0396] Step 3:

[0397] User

[0398] Access the platform, enter the required personal information to create an account, and after logging in, access the skill test interface provided by the server.

[0399] Step 4:

[0400] server

[0401] The system verifies the user's login information and displays the appropriate skills test on the screen. The user is then ready to begin the skills test.

[0402] Step 5:

[0403] Terminal

[0404] As the user starts the skill test, the system provides an input interface, transmits the user's input data to the server in real time, and collects the candidate's biometric data, which is also transmitted to the server.

[0405] Step 6:

[0406] User

[0407] Follow the on-screen instructions to complete the skills test, answering each task in turn and providing additional information as needed.

[0408] Step 7:

[0409] server

[0410] The system analyzes the skill test response data received in real time to quantify the candidate's skill level, while analyzing the biometric data sent from the device to evaluate the candidate's stress level and concentration.

[0411] Step 8:

[0412] server

[0413] Skill assessment and biometric assessment are combined to generate an overall score for the candidate, which reflects the candidate's skills, aptitudes, and personality.

[0414] Step 9:

[0415] server

[0416] Based on the generated overall score, the system compares the results with the company's requirements to create a list of the most suitable candidates, and provides information about the candidates to the company.

[0417] Step 10:

[0418] Terminal

[0419] It displays a detailed report of completed skill tests and assessment results to users, allowing them to receive feedback on their skills and aptitudes.

[0420] Step 11:

[0421] company

[0422] The company reviews the list of candidates provided by the server and plans interviews or additional evaluations as the next step. Finally, the company hires the best candidate.

[0423] Example 1

[0424] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0425] Traditional hiring processes are prone to subjective evaluation and bias, making it difficult to conduct fair evaluations. It's also difficult to accurately assess applicants' skills and aptitude, making it difficult to hire the best candidates. Furthermore, important factors like a candidate's stress level and level of concentration tend to be overlooked, resulting in cases where candidates don't accurately match the talent specifications companies are looking for.

[0426] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0427] In this invention, the server includes a means for storing the skill sets, aptitude criteria, and evaluation criteria provided by the company in a database, a means for analyzing the skill test results and biometric data in real time to generate an overall score for the candidate, and a means for integrating the skill evaluation and biometric evaluation to generate an overall score. This eliminates subjective bias and enables the evaluation of candidates in an objective and fair manner. Factors such as stress level and concentration are also taken into account, making it possible to list the most suitable candidates who meet the company's desired criteria.

[0428] "Biometric data" refers to data that measures a candidate's physical characteristics and is stored digitally, such as facial expressions, tone of voice, and heart rate.

[0429] A "skills test" is a test administered to assess a candidate's specific abilities or knowledge. Examples include programming tests and tasks that assess problem-solving ability.

[0430] The "total score" is an evaluation of a candidate calculated based on the results of the skill test and biometric data. It is used to comprehensively judge the candidate's abilities and aptitude.

[0431] The "database" is an information management system for storing the skill sets, aptitude conditions, and evaluation criteria provided by the company. Specifically, a database management system such as MySQL is used.

[0432] "Real-time analysis" is the process of analyzing skill test results and biometric data as they are collected, allowing for rapid candidate evaluation.

[0433] "A means of listing the best candidates based on a company's requirements" is a method for selecting candidates with a high overall score based on evaluation criteria set by the company in advance. This method allows companies to efficiently identify the talent they are looking for.

[0434] "Cameras, microphones, and sensors" are hardware devices used to collect biometric data on candidates, including facial expressions, tone of voice, and heart rate.

[0435] "Means for quantifying skill levels" refers to a method for expressing a candidate's specific abilities as a number based on the results of a skill test. This allows for an objective evaluation of the candidate's abilities.

[0436] This invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. This system performs various processes based on the roles of the server, terminal, and user.

[0437] Server Operation

[0438] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by companies in a database. This allows candidates to be evaluated based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate. This creates a foundation for evaluating candidates in a standardized manner.

[0439] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the device to evaluate the candidate's stress level and concentration. Based on this, the server combines the skill assessment and biometric assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0440] Specific hardware and software features include a server running Python and a MySQL database for data processing and analysis, and TensorFlow and Keras machine learning algorithms for analyzing biometric data.

[0441] Device behavior

[0442] The terminal is the device used by the candidate and provides the interface for performing the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. The terminal also uses cameras, microphones, and sensors to collect biometric data and transmits the data to the server. This data includes the candidate's facial expressions, tone of voice, heart rate, etc.

[0443] Specific hardware such as cameras, microphones, and sensors are required to accurately collect biometric data, which can be used to assess a candidate's stress level, concentration, etc. in real time.

[0444] User Actions

[0445] The user is a candidate and creates an account by entering the necessary personal information on the platform when first accessing it. After logging in, the user takes a skill test provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user checks the results and understands their strengths and areas for improvement.

[0446] Specific examples

[0447] For example, if a company sets "programming ability" and "stress management ability" as requirements, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers, as well as facial expressions and tone of voice, and sends them to the server.

[0448] The server scores Candidate A's programming skills and evaluates his / her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's desired criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's criteria.

[0449] Prompt Sentence Examples

[0450] An example prompt is:

[0451] "What methods and techniques will the system use to assess candidates' programming and stress management abilities?"

[0452] In this way, the present invention eliminates bias and ensures an objective and fair hiring process.

[0453] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0454] Step 1: Register your company information

[0455] The server stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. The input is information provided by the company (e.g., programming ability, teamwork ability). The server stores this information in the database, and the company's conditions are registered in the database as output.

[0456] Step 2: Register for the skills test

[0457] The server registers the skill tests and assignments provided by the company in a database. The input is the skill tests and assignments (e.g. Java programming tests) provided by the company. The server saves these in the database, and the registered skill tests and assignments are stored in the database as output.

[0458] Step 3: Create a candidate account

[0459] A user creates an account by entering the necessary personal information when first accessing the site. The input includes the personal information provided by the user (e.g., name, email address, password). The server stores this information in a database, and the output is a user account.

[0460] Step 4: Start the skills test

[0461] After logging in, the user starts the skill test provided by the server. The input is the user's login information and a command to start the test. The server starts the skill test based on this, and the test screen is displayed on the terminal as output.

[0462] Step 5: Perform a skills test

[0463] The user takes the skill test by following the instructions displayed on the screen. The input is each question in the test that is displayed on the screen. The user answers the questions, and the device sends the answers to the server in real time. The output is the answer data sent to the server.

[0464] Step 6: Collecting Biometric Data

[0465] The terminal collects biometric data using cameras, microphones, and sensors. The inputs include data such as the candidate's facial expression, tone of voice, and heart rate. The terminal collects this data and sends it to the server. The output is the biometric data sent to the server.

[0466] Step 7: Analyze the data

[0467] The server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data to evaluate the candidate's stress level and concentration. The inputs are the response data and biometric data. The server analyzes the data using Python scripts and TensorFlow, and the output is the evaluation results.

[0468] Step 8: Generate an overall score

[0469] The server combines the skill assessment and biometric assessment to generate an overall score. The inputs are the results of the skill assessment and biometric assessment. The server combines these assessments and calculates an overall score as output.

[0470] Step 9: Shortlist the best candidates

[0471] The server compares the overall score with the company's requirements and lists the most suitable candidates. The input is the overall score and the company's requirements. The server compares these and generates a list of the most suitable candidates as output.

[0472] Step 10: Providing the best candidates

[0473] The server provides companies with the best candidates. As input, it has the information of the candidates listed. The server provides this to the company and as output, it provides the information in a format that the company can view.

[0474] This process allows the system to eliminate bias and ensure an objective and fair hiring process.

[0475] (Application example 1)

[0476] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0477] Traditional recruitment systems can be biased when evaluating candidates' skills, aptitude, and personality, making it difficult to evaluate them fairly. Furthermore, systems that evaluate the effectiveness of advertising are unable to incorporate real-time responses from viewers, making it difficult to provide personalized advertising. For these reasons, there is a need for more accurate and objective evaluation systems in both the fields of candidate evaluation and ad personalization.

[0478] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0479] In this invention, the server includes means for collecting biometric data to evaluate the skills, aptitude, and personality of candidates, means for the candidate to take a skill test according to instructions displayed on the screen, means for analyzing the skill test results and biometric data in real time to generate an overall score for the candidate, means for compiling a list of optimal candidates based on conditions preset by the company, means for providing the optimal candidates to the company, means for collecting biometric data of advertisement viewers and personalizing advertisements, and means for measuring the effectiveness of advertisements in real time and providing feedback on the results, thereby enabling fair and efficient candidate evaluation and highly accurate advertisement personalization.

[0480] A "candidate" is an individual with specific skills and aptitudes who applies for a role or position sought by a company.

[0481] "Biometric data" is data that digitally records an individual's physiological or behavioral characteristics, such as heart rate, facial expression, and tone of voice.

[0482] A "skills test" is an exam or task designed to assess a candidate's specific knowledge or abilities.

[0483] "Means of real-time analysis" refers to technologies and devices that process data immediately and reflect the results instantly.

[0484] "Composite score" is a number or indicator that indicates the candidate's overall evaluation, generated by integrating the results of the skill test and biometric data.

[0485] "Conditions set in advance by the company" are the standards set in advance by the company regarding the skills, aptitudes, and other requirements that the company requires of job candidates.

[0486] A "means of shortlisting the best candidates" is a method or technique for selecting the most suitable candidates based on set criteria.

[0487] An "advertising viewer" is an individual who views an advertisement and is the person to whom the advertisement is directed.

[0488] "Means for personalizing advertising" are methods and technologies for individually tailoring advertising content based on the viewer's characteristics, interests, and responses.

[0489] "Means for measuring the effectiveness of advertising in real time and providing feedback on the results" refers to a system that instantly analyzes the reactions of viewers of advertisements and reflects the results in advertising strategies.

[0490] This invention is a system that objectively evaluates candidates' skills, aptitude, and personality to realize a fair hiring process. The system includes a server, a terminal, and a user. Furthermore, the system can collect responses from advertisement viewers, personalize advertisements, and measure their effectiveness.

[0491] Server Operation

[0492] The server is responsible for storing the desired skills, aptitudes, and evaluation criteria provided by companies in a database. Based on the conditions set by the company, the server manages the candidates' skill tests and assignments and generates an evaluation profile for each candidate.

[0493] When a candidate begins a skill test, the server receives and analyzes the skill test response data and biometric data in real time. This analysis is primarily performed using software such as OpenCV, TensorFlow, and Keras. This integrates the candidate's skill assessment and biometric assessment to generate an overall score. In addition, the server also collects biometric data of ad viewers, measures the effectiveness of personalized advertising, and provides feedback on the results.

[0494] Device behavior

[0495] The terminal is a device used by candidates and ad viewers, and provides an interface for taking skill tests and viewing ads. The terminal is equipped with a camera, microphone, and various sensors, and these devices are used to collect biometric data.

[0496] During the skill test, the device transmits the candidate's input in real time to a server. The device also collects biometric information such as facial expressions, tone of voice, and heart rate of the ad viewer and transmits this information to the server, which allows for personalized advertising.

[0497] User Actions

[0498] Users are candidates or ad viewers, and when they first access the platform, they create an account by entering the necessary personal information. After logging in, candidates take a skill test provided by the server. During the skill test, users answer questions in order by following the instructions displayed on the screen. Once the skill test is complete, users can check the results and identify their strengths and areas for improvement.

[0499] Ad viewers' responses are collected in real time as they watch the ad, and based on their responses, ad content is personalized and reflected in future viewings.

[0500] Specific examples

[0501] For example, if a company sets "programming ability" and "stress management ability" as required criteria, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers as well as facial expressions and tone of voice, and sends these to the server. The server then scores Candidate A's programming skills and evaluates his or her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's required criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0502] In addition, when an ad viewer is viewing a specific ad, their reaction can be collected and analyzed in real time and used to determine the next ad display. A specific example of a prompt sentence could be entered as follows:

[0503] "We want to develop a system that analyzes users' emotional responses and stress levels in real time, and then delivers personalized advertising content that is optimal for them. We will collect and analyze biometric data such as facial expressions, heart rate, and tone of voice, and use this data to evaluate the effectiveness of advertising. As part of ad personalization, we want to reflect content that the viewer is particularly interested in when displaying ads next time."

[0504] In this way, the present invention improves the fairness of candidate evaluation and enables advertising personalization and effectiveness measurement.

[0505] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0506] Step 1:

[0507] The server stores the desired skill sets, aptitudes, and evaluation criteria provided by companies in a database. This creates a foundation for evaluating candidates based on each company's unique criteria. The input is the requirements data provided by the company, and the output is the evaluation criteria stored in the database.

[0508] Step 2:

[0509] The server registers various skill tests and assignments in a database and generates an individual assessment profile for each candidate. The input is the test question data, and the output is the assessment profile for each candidate. This data is used for subsequent analysis.

[0510] Step 3:

[0511] The candidate starts the skill test using the terminal. The terminal sends the candidate's input to the server in real time. The input is the candidate's answer data, and the output is the server that received the data. The server prepares to analyze the received data immediately.

[0512] Step 4:

[0513] The device uses cameras, microphones, and sensors to collect biometric data (facial expressions, tone of voice, heart rate, etc.) of the candidate and transmits it to a server. The input is the collected biometric data, and the output is the data transmitted to the server.

[0514] Step 5:

[0515] The server analyzes the skill test response data and biometric data in real time. Software such as OpenCV, TensorFlow, and Keras is used for the analysis. The input is the response data and biometric data, and the output is the analysis results. Specific operations include facial expression recognition using models, voice tone analysis, and heart rate variability analysis.

[0516] Step 6:

[0517] The server generates an overall score for the candidate based on the analysis results. The input is the analyzed data, and the output is the overall score, which integrates the candidate's skills and stress assessment.

[0518] Step 7:

[0519] The server compares the overall score with the company's set conditions and lists the most suitable candidates. The input is the overall score and the company's conditions, and the output is a list of the most suitable candidates.

[0520] Step 8:

[0521] The server provides companies with the best candidates. The input is the candidate information listed, and the output is the candidate information sent to the company.

[0522] Step 9:

[0523] The ad viewer watches the ad using a device. The device uses a camera, microphone, and sensors to collect the viewer's responses and send them to a server. The input is biometric data from the time the ad was viewed, and the output is the biometric data sent to the server.

[0524] Step 10:

[0525] The server analyzes the biometric data of the ad viewers in real time to measure the effectiveness of the ad. The input is the viewer's biometric data, and the output is the analysis result of the ad's effectiveness. This analysis includes evaluating emotional responses and measuring stress levels.

[0526] Step 11:

[0527] The server personalizes the ads based on the analysis results and provides the most suitable ads to the viewer. The input is the analysis result of the advertising effectiveness, and the output is personalized advertising content. This result is reflected in subsequent viewings.

[0528] Examples:

[0529] For example, if a viewer shows strong interest in a particular ad, the next ad display can be optimized based on that reaction data. This is achieved using technologies such as "generative AI models and prompt sentences."

[0530] Example prompt sentence:

[0531] "We want to develop a system that analyzes users' emotional responses and stress levels in real time, and then delivers personalized advertising content that is optimal for them. We will collect and analyze biometric data such as facial expressions, heart rate, and tone of voice, and use this data to evaluate the effectiveness of advertising. As part of ad personalization, we want to reflect content that the viewer is particularly interested in when displaying ads next time."

[0532] In this way, a system is built that achieves high accuracy and fairness in both candidate evaluation and ad personalization.

[0533] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0534] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. This system includes a means for incorporating an emotion engine to recognize candidates' emotions from biometric data and reflect them in the evaluation.

[0535] Server Operation

[0536] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. This allows evaluation criteria to be set based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate.

[0537] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the terminal and recognizes the candidate's emotions using an emotion engine. Based on this, the server integrates the skill assessment, biometric assessment, and emotion assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0538] Device behavior

[0539] The terminal is the device used by the candidate and provides the interface for taking the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. At the same time, the terminal uses a camera, microphone, and sensors to collect biometric data, which is then transmitted to the server. This biometric data includes facial expressions, tone of voice, heart rate, etc.

[0540] User Actions

[0541] The user is a candidate and creates an account by entering the required personal information on the platform when first accessing it. After logging in, the user accesses the skill test interface provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user can check the results and understand their strengths and areas for improvement.

[0542] Specific examples

[0543] For example, if a company sets the requirements for a candidate as "programming ability," "stress management ability," and "emotional stability," the system works as follows:

[0544] Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers as well as facial expressions and tone of voice, and sends these to the server. The emotion engine analyzes the collected data, recognizes Candidate A's emotions in real time, and evaluates their stress level and concentration.

[0545] The server scores Candidate A's programming skills and generates an overall score by combining the emotional evaluation from the emotion engine and the biometric evaluation. Based on this overall score, the server compares the candidates with the company's requirements and creates a list of the most suitable candidates. The company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0546] In this way, the present invention eliminates bias and provides an objective and fair hiring process, allowing companies to effectively select top talent.

[0547] The processing flow will be explained below.

[0548] Step 1:

[0549] server

[0550] The desired skill sets, aptitude conditions, and evaluation criteria provided by companies are stored in a database, which allows evaluation criteria to be set based on each company's unique conditions.

[0551] Step 2:

[0552] Terminal

[0553] Activate and verify that cameras, microphones, and sensors are working properly, so they are ready to accurately collect biometric and emotional data.

[0554] Step 3:

[0555] User

[0556] Access the platform, enter the required personal information to create an account, and after logging in, access the skill test interface provided by the server.

[0557] Step 4:

[0558] server

[0559] The system verifies the user's login information and displays the appropriate skills test on the screen. The user is then ready to begin the skills test.

[0560] Step 5:

[0561] Terminal

[0562] As the user starts the skill test, the system provides an input interface and transmits the user's input data to the server in real time. It also collects biometric data using cameras, microphones, and sensors and transmits this data to the server.

[0563] Step 6:

[0564] User

[0565] Follow the on-screen instructions to complete the skills test, answering each task in turn and providing additional information as needed.

[0566] Step 7:

[0567] server

[0568] The system analyzes the skill test response data received in real time and quantifies the candidate's skill level. At the same time, it uses an emotion engine to analyze the biometric data sent from the device and recognize the candidate's emotions. Based on the recognized emotional data, it evaluates the candidate's stress level and concentration level.

[0569] Step 8:

[0570] server

[0571] Skill assessment, emotional assessment, and biometric assessment are combined to generate an overall score for the candidate, which reflects the candidate's skills, aptitudes, emotions, and personality.

[0572] Step 9:

[0573] server

[0574] Based on the generated overall score, the system compares the results with the company's requirements to create a list of the most suitable candidates, and provides information about the candidates to the company.

[0575] Step 10:

[0576] Terminal

[0577] It displays a detailed report of completed skill tests and assessment results to users, allowing them to receive feedback on their skills and aptitudes.

[0578] Step 11:

[0579] company

[0580] The company reviews the list of candidates provided by the server and plans interviews or additional evaluations as the next step. Finally, the company hires the best candidate.

[0581] Example 2

[0582] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0583] In the traditional recruitment process, it is difficult to objectively evaluate a candidate's skills, aptitude, and personality, and the evaluator's subjectivity and bias are easily incorporated, making it difficult to conduct a fair evaluation and select the right talent.In addition, because the evaluation does not take into account the candidate's emotions or stress levels, it is difficult to accurately predict how a candidate will perform in a real work environment.

[0584] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0585] In this invention, the server includes means for storing the skill sets, aptitude conditions, and evaluation criteria provided by the company in a database, means for registering various skill tests and tasks in the database and generating an individual evaluation profile for each candidate, means for receiving and analyzing candidate skill test response data in real time, means for analyzing biometric data and recognizing the candidate's emotions using an emotion engine, means for generating an overall score by integrating the skill evaluation, biometric evaluation, and emotion evaluation, means for comparing the overall score with the company's conditions to list the most suitable candidates, and means for providing the most suitable candidates to the company. This makes it possible to objectively evaluate the skills, aptitude, and personality of candidates and realize a fair hiring process that eliminates bias.

[0586] "Company conditions" refers to the skill sets, aptitudes, and evaluation criteria required by the company.

[0587] "Database" refers to a system for storing and managing data such as information provided by companies, skill tests, assignments, and candidate evaluation profiles.

[0588] A "skills test" refers to a series of questions or tasks administered to assess a candidate's abilities or skills.

[0589] "Biometric data" refers to a candidate's biological information, specifically facial expressions, tone of voice, heart rate, etc.

[0590] An "emotion engine" refers to software or algorithms that analyze collected biometric data and recognize a candidate's emotions.

[0591] "Skills assessment" refers to a method of quantifying the results of a candidate's skills test and evaluating their abilities.

[0592] "Biometric assessment" refers to a method of assessing a candidate's status or characteristics based on collected biometric data.

[0593] "Emotional assessment" refers to an assessment based on the candidate's emotional state recognized using an emotion engine.

[0594] "Composite Score" refers to a candidate's overall assessment score generated by integrating skill assessment, biometric assessment, and emotional assessment.

[0595] The "best candidate" is the candidate who best meets the company's requirements.

[0596] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. The system includes a means for incorporating candidates' emotions into the evaluation by combining an emotion engine with analysis of biometric data.

[0597] Server Operation

[0598] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. Specifically, by registering the information provided by the company in the database, each company's unique criteria are saved. The server also registers skill tests and assignments in the database. Processing is then carried out to generate an individual evaluation profile for each candidate.

[0599] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, the server receives biometric data sent from the device. This biometric data includes facial expressions, tone of voice, heart rate, etc. The server uses an emotion engine to analyze the biometric data and recognize the candidate's emotions.

[0600] The server combines the skill, biometric, and emotional assessments to generate a comprehensive score, which is then matched with the company's requirements to create a list of the most suitable candidates and provide them to the company.

[0601] Device behavior

[0602] The terminal is the device used by the candidate and provides the interface for the skills test. When the candidate starts the skills test, the terminal transmits answer data in real time to the server, including all data entered by the candidate.

[0603] At the same time, the device collects biometric data using cameras, microphones, and sensors, such as a webcam, microphone, and heart rate sensor, and transmits this biometric data to a server to assess the candidate's emotional state.

[0604] User Actions

[0605] The user is a candidate, and when accessing the platform for the first time, they create an account by entering the necessary personal information. After logging in, they access the skill test interface provided by the server. The candidate answers the questions by following the instructions displayed on the screen.

[0606] Once the skills test is complete, users can review their results to identify their strengths and areas for improvement.

[0607] Specific examples

[0608] For example, if a company sets requirements for "programming skills," "stress management skills," and "emotional stability," the system works as follows:

[0609] Candidate A logs into the platform and takes a programming skills test.

[0610] During the test, the device collects Candidate A's answers as well as his / her facial expressions and tone of voice, and sends them to the server.

[0611] The emotion engine analyzes the collected data, recognizes Candidate A's emotions in real time, and assesses his stress level and concentration.

[0612] The server scores Candidate A's programming skills, and generates an overall score by combining the emotional evaluation by the emotion engine and the biometric evaluation. Based on this overall score, the server compares the candidates with the company's requirements and creates a list of the most suitable candidates.

[0613] The company checks the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0614] In this way, the present invention eliminates bias and provides an objective and fair hiring process, allowing companies to effectively select top talent.

[0615] Example prompts to input to the generative AI model

[0616] System Overview: A system that objectively evaluates candidates' skills, aptitude, and personality to ensure a fair hiring process.

[0617] Technologies used: Emotion engine, biometric data analysis

[0618] Example: A company sets evaluation criteria such as "programming ability," "stress management ability," and "emotional stability."

[0619] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0620] Step 1: Store company information in a database

[0621] The server receives the skill sets, aptitude conditions, and evaluation criteria provided by the companies and stores them in a database.

[0622] Input: Skill set, qualifications, and evaluation criteria provided by the company

[0623] Output: Company condition information stored in the database

[0624] Specific operation: The server acquires skill sets such as "Java programming," "project management," and "communication skills" sent by Company A and stores them in a database called "Company A_Evaluation Criteria."

[0625] Step 2: Register for the Skills Test and Assignment

[0626] The server registers various skill tests and assignments in a database and generates an evaluation profile for each candidate.

[0627] Input: Skill test, assignment information

[0628] Output: Assessment profile for each candidate

[0629] Specific operation: The server registers the "Java Programming Proficiency Test" and "Project Presentation Assignment" in the database and creates a "Candidate A_Evaluation Profile" for Candidate A.

[0630] Step 3: Initiating the skill test and receiving response data

[0631] When a candidate starts a skill test, the device sends answer data to the server in real time.

[0632] Input: Candidate's answer data

[0633] Output: Answer data sent to the server

[0634] Specific operation: Every time the user clicks "Next Question," the device sends the answer data to the server in real time.

[0635] Step 4: Collect and analyze biometric data

[0636] The device uses a camera, microphone, and sensors to collect biometric data and transmits it to a server, which then uses an emotion engine to analyze the biometric data and recognize the candidate's emotions.

[0637] Input: Biometric data (facial expression, tone of voice, heart rate)

[0638] Output: Parsed emotion data

[0639] Specific operation: The device captures facial expressions with a camera, records voice tone with a microphone, and measures heart rate with a heart rate sensor, and sends this data to a server. The emotion engine analyzes the received data and calculates stress levels and concentration levels.

[0640] Step 5: Integrate the data and generate an overall score

[0641] The server combines the skill assessment, biometric assessment, and emotional assessment to generate an overall score.

[0642] Input: Skill assessment data, Biometric assessment data, Emotion assessment data

[0643] Output: Overall score

[0644] Specific operation: The server calculates a skill score (e.g., "85 points / 100 points") based on the answer data, and also performs emotional evaluation using an emotion engine and biometric evaluation, integrating these to generate "Candidate A_Overall Score" (e.g., "Score: 90 / 100").

[0645] Step 6: Matching with companies and providing candidate lists

[0646] The server compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0647] Input: Overall score, company conditions

[0648] Output: A list of the best candidates

[0649] Specific operation: The server compares "Candidate A_Total Score" with the conditions of Company A, determines from the comparison results that "Candidate A meets the conditions of Company A" and adds him to the list. The company then checks this list and selects the most suitable candidate.

[0650] (Application example 2)

[0651] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0652] In the modern hiring process, it is difficult to fairly evaluate a candidate's skills, aptitude, and personality. Furthermore, these evaluations tend to rely on subjective judgment and are prone to bias. For candidates working in retail stores, evaluating their customer service attitude and stress tolerance is particularly important, but traditional methods make these evaluations difficult. Furthermore, there is a lack of technology that can grasp a candidate's emotional state in real time and reflect that information in the overall evaluation.

[0653] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting biometric data to evaluate the skills, aptitude, and personality of candidates; means for the candidate to take a skill test according to instructions displayed on the screen; means for analyzing the skill test results and biometric data in real time and generating an overall score for the candidate; means for comparing the results with registered evaluation criteria and listing the candidates who best meet the conditions; means for providing the most appropriate candidates to the company; means for analyzing facial expressions and voice in real time using a camera, microphone, and sensor to evaluate the candidate's emotions and stress level; and means for presenting the evaluation results using an overall score based on the customer service attitude and stress tolerance set by the company. This enables a comprehensive evaluation of the candidate's skills and emotional state and a fair and comprehensive judgment.

[0654] A "candidate" is a person who is being evaluated during the recruitment process.

[0655] "Skills" are specific skills or abilities that a candidate possesses.

[0656] "Aptitude" refers to a candidate's ability to fit into a particular job or environment.

[0657] "Personality" refers to the candidate's character and behavioral traits.

[0658] "Biometric data" refers to data based on a candidate's biological information, such as facial expression analysis, voice analysis, and heart rate.

[0659] A "skills test" is an examination to evaluate a candidate's skills and abilities.

[0660] The "total score" is an evaluation index calculated by integrating the results of the skill test and biometric data.

[0661] The "emotion recognition model" is an artificial intelligence model that recognizes candidates' emotions in real time from biometric data.

[0662] "Real-time analysis" is the process of processing data in real time and outputting the results immediately.

[0663] "Evaluation criteria" are standard values ​​used to quantify the skills and aptitudes that companies are looking for.

[0664] "Emotional state" refers to the candidate's current emotional state, including emotions such as joy, anger, sadness, and happiness.

[0665] "Stress tolerance" refers to a candidate's ability to resist and adapt to stress.

[0666] "Customer service attitude" refers to the attitude and behavior displayed by the candidate when performing customer service duties.

[0667] "Company" refers to the organization or entity seeking to recruit a Candidate.

[0668] The present invention provides a system for objectively evaluating a candidate's skills, aptitude, and personality to realize a fair hiring process. Specific embodiments are described below.

[0669] Server Operation

[0670] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. This allows evaluation criteria to be set based on each company's unique conditions. Next, the server registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate.

[0671] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it processes the biometric data sent from the device and recognizes the candidate's emotions using an emotion recognition model. Based on this, the server integrates the skill assessment, biometric assessment, and emotion assessment to generate an overall score. Finally, it compares this overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0672] Device behavior

[0673] The terminal is the device used by the candidate and provides the interface for taking the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. At the same time, the terminal uses a camera, microphone, and sensors to collect biometric data, which is then transmitted to the server. This biometric data includes facial expressions, tone of voice, heart rate, etc.

[0674] User Actions

[0675] The user is a candidate and creates an account by entering the required personal information on the platform when first accessing it. After logging in, the user accesses the skill test interface provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user can check the results and understand their strengths and areas for improvement.

[0676] Hardware and software used

[0677] The hardware used in implementing this invention includes a smartphone, smart glasses, and a webcam. The software used includes OpenCV, Dlib, Keras (with TensorFlow backend), Python, and emotion recognition models. This combination of hardware and software allows for real-time data collection, analysis, and display.

[0678] Specific examples

[0679] For example, if a candidate is considering a position at a high-end restaurant in Tokyo, the system will evaluate the candidate's smile, nervousness, and friendliness in real time during the interview. In this example, the following prompts are used:

[0680] "Among your friends and family, what are the characteristics of people you find particularly approachable? Please describe specifically what their facial expressions are like."

[0681] The prompts allow candidates to be assessed on their emotional stability and customer service attitude, resulting in an overall score, enabling companies to ensure an objective and fair hiring process and effectively select the best candidates.

[0682] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0683] Step 1:

[0684] When a user first accesses the platform, they create an account by entering the necessary personal information, including basic information such as name, contact details, and resume, and the server receives this data and stores it in a database.

[0685] Step 2:

[0686] The user logs in and accesses the skill test interface provided by the server. Based on the login information, the server generates a skill test interface dedicated to the user and displays it on the screen.

[0687] Step 3:

[0688] The user follows the on-screen instructions to begin the skill test, and each response is sent in real time from the device to the server, which analyzes the response data and monitors the user's progress.

[0689] Step 4:

[0690] The device will begin collecting biometric data about the user using cameras, microphones, and sensors, including facial expressions, tone of voice, and heart rate, in real time, and will send this data to a server.

[0691] Step 5:

[0692] The server inputs the received biometric data into an emotion recognition model to analyze the user's emotional state. The emotion recognition model uses facial expression recognition and voice analysis algorithms to assess the user's current emotions and stress levels.

[0693] Step 6:

[0694] The server combines the skill test results with the emotional assessment obtained from biometric data to generate an overall score. Skill assessment is based on the accuracy rate and speed of answers, while emotional assessment is based on emotional stability and stress tolerance.

[0695] Step 7:

[0696] The server compares the overall score with the evaluation criteria set by the company and lists the candidates who best fit the criteria. This determines how well the candidate's overall evaluation matches the company's requirements.

[0697] Step 8:

[0698] The server provides companies with the best candidates and allows them to view the assessment results on a dashboard, including detailed skill test results, biometric assessment, and overall score.

[0699] Step 9:

[0700] Based on the results of the assessment, companies can decide on interviews or next steps in the selection process. Using the dashboard, company personnel can view detailed assessments of candidates and input further questions and prompts into the generative AI model for further assessment.

[0701] For example, the following prompt text can be used:

[0702] "Among your friends and family, what are the characteristics of people you find particularly approachable? Please describe specifically what their facial expressions are like."

[0703] This process allows companies to comprehensively evaluate a candidate's skills and emotional state, allowing them to make a fair and comprehensive decision.

[0704] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0705] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0706] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0707] [Third embodiment]

[0708] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0709] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0710] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0711] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0712] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0713] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0714] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0715] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0716] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0717] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0718] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0719] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0720] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. In this system, various processes are performed based on the roles of the server, terminal, and user.

[0721] Server Operation

[0722] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by companies in a database. This allows candidates to be evaluated based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate. This creates a foundation for evaluating candidates in a standardized manner.

[0723] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the device to evaluate the candidate's stress level and concentration. Based on this, the server combines the skill assessment and biometric assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0724] Device behavior

[0725] The terminal is the device used by the candidate and provides the interface for performing the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. The terminal also uses cameras, microphones, and sensors to collect biometric data and transmits the data to the server. This data includes the candidate's facial expressions, tone of voice, heart rate, etc.

[0726] User Actions

[0727] The user is a candidate and creates an account by entering the necessary personal information on the platform when first accessing it. After logging in, the user takes a skill test provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user checks the results and understands their strengths and areas for improvement.

[0728] Specific examples

[0729] For example, if a company sets "programming ability" and "stress management ability" as requirements, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers, as well as facial expressions and tone of voice, and sends them to the server.

[0730] The server scores Candidate A's programming skills and evaluates his / her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's desired criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's criteria.

[0731] In this way, the present invention eliminates bias and ensures an objective and fair hiring process.

[0732] The processing flow will be explained below.

[0733] Step 1:

[0734] server

[0735] The desired skill sets, aptitude conditions, and evaluation criteria provided by companies are stored in a database, which allows evaluation criteria to be set based on each company's unique conditions.

[0736] Step 2:

[0737] Terminal

[0738] Activate and verify that your cameras, microphones, and sensors are working properly, so they are ready to accurately collect biometric data.

[0739] Step 3:

[0740] User

[0741] Access the platform, enter the required personal information to create an account, and after logging in, access the skill test interface provided by the server.

[0742] Step 4:

[0743] server

[0744] The system verifies the user's login information and displays the appropriate skills test on the screen. The user is then ready to begin the skills test.

[0745] Step 5:

[0746] Terminal

[0747] As the user starts the skill test, the system provides an input interface and transmits the user's input data to the server in real time, and also collects the candidate's biometric data and transmits it to the server.

[0748] Step 6:

[0749] User

[0750] Follow the on-screen instructions to complete the skills test, answering each task in turn and providing additional information as needed.

[0751] Step 7:

[0752] server

[0753] The system analyzes the skill test response data received in real time to quantify the candidate's skill level, while analyzing the biometric data sent from the device to evaluate the candidate's stress level and concentration.

[0754] Step 8:

[0755] server

[0756] Skill assessment and biometric assessment are combined to generate an overall score for the candidate, which reflects the candidate's skills, aptitudes, and personality.

[0757] Step 9:

[0758] server

[0759] Based on the generated overall score, the system compares the results with the company's requirements to create a list of the most suitable candidates, and provides information about the candidates to the company.

[0760] Step 10:

[0761] Terminal

[0762] It displays a detailed report of completed skill tests and assessment results to users, allowing them to receive feedback on their skills and aptitudes.

[0763] Step 11:

[0764] company

[0765] The company reviews the list of candidates provided by the server and plans interviews or additional evaluations as the next step. Finally, the company hires the best candidate.

[0766] Example 1

[0767] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0768] Traditional hiring processes are prone to subjective evaluation and bias, making it difficult to conduct fair evaluations. It's also difficult to accurately assess applicants' skills and aptitude, making it difficult to hire the best candidates. Furthermore, important factors like a candidate's stress level and level of concentration tend to be overlooked, resulting in cases where candidates don't accurately match the talent specifications companies are looking for.

[0769] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0770] In this invention, the server includes a means for storing the skill sets, aptitude criteria, and evaluation criteria provided by the company in a database, a means for analyzing the skill test results and biometric data in real time to generate an overall score for the candidate, and a means for integrating the skill evaluation and biometric evaluation to generate an overall score. This eliminates subjective bias and enables the evaluation of candidates in an objective and fair manner. Factors such as stress level and concentration are also taken into account, making it possible to list the most suitable candidates who meet the company's desired criteria.

[0771] "Biometric data" refers to data that measures a candidate's physical characteristics and is stored digitally, such as facial expressions, tone of voice, and heart rate.

[0772] A "skills test" is a test administered to assess a candidate's specific abilities or knowledge. Examples include programming tests and tasks that assess problem-solving ability.

[0773] The "total score" is an evaluation of a candidate calculated based on the results of the skill test and biometric data. It is used to comprehensively judge the candidate's abilities and aptitude.

[0774] The "database" is an information management system for storing the skill sets, aptitude conditions, and evaluation criteria provided by the company. Specifically, a database management system such as MySQL is used.

[0775] "Real-time analysis" is the process of analyzing skill test results and biometric data as they are collected, allowing for rapid candidate evaluation.

[0776] "A means of listing the best candidates based on a company's requirements" is a method for selecting candidates with a high overall score based on evaluation criteria set by the company in advance. This method allows companies to efficiently identify the talent they are looking for.

[0777] "Cameras, microphones, and sensors" are hardware devices used to collect biometric data on candidates, including facial expressions, tone of voice, and heart rate.

[0778] "Means for quantifying skill levels" refers to a method for expressing a candidate's specific abilities as a number based on the results of a skill test. This allows for an objective evaluation of the candidate's abilities.

[0779] This invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. This system performs various processes based on the roles of the server, terminal, and user.

[0780] Server Operation

[0781] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by companies in a database. This allows candidates to be evaluated based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate. This creates a foundation for evaluating candidates in a standardized manner.

[0782] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the device to evaluate the candidate's stress level and concentration. Based on this, the server combines the skill assessment and biometric assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0783] Specific hardware and software features include a server running Python and a MySQL database for data processing and analysis, and TensorFlow and Keras machine learning algorithms for analyzing biometric data.

[0784] Device behavior

[0785] The terminal is the device used by the candidate and provides the interface for performing the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. The terminal also uses cameras, microphones, and sensors to collect biometric data and transmits the data to the server. This data includes the candidate's facial expressions, tone of voice, heart rate, etc.

[0786] Specific hardware such as cameras, microphones, and sensors are required to accurately collect biometric data, which can be used to assess a candidate's stress level, concentration, etc. in real time.

[0787] User Actions

[0788] The user is a candidate and creates an account by entering the necessary personal information on the platform when first accessing it. After logging in, the user takes a skill test provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user checks the results and understands their strengths and areas for improvement.

[0789] Specific examples

[0790] For example, if a company sets "programming ability" and "stress management ability" as requirements, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers, as well as facial expressions and tone of voice, and sends them to the server.

[0791] The server scores Candidate A's programming skills and evaluates his / her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's desired criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's criteria.

[0792] Prompt Sentence Examples

[0793] An example prompt is:

[0794] "What methods and techniques will the system use to assess candidates' programming and stress management abilities?"

[0795] In this way, the present invention eliminates bias and ensures an objective and fair hiring process.

[0796] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0797] Step 1: Register your company information

[0798] The server stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. The input is information provided by the company (e.g., programming ability, teamwork ability). The server stores this information in the database, and the company's conditions are registered in the database as output.

[0799] Step 2: Register for the skills test

[0800] The server registers the skill tests and assignments provided by the company in a database. The input is the skill tests and assignments (e.g. Java programming tests) provided by the company. The server saves these in the database, and the registered skill tests and assignments are stored in the database as output.

[0801] Step 3: Create a candidate account

[0802] A user creates an account by entering the necessary personal information when first accessing the site. The input includes the personal information provided by the user (e.g., name, email address, password). The server stores this information in a database, and the output is a user account.

[0803] Step 4: Start the skills test

[0804] After logging in, the user starts the skill test provided by the server. The input is the user's login information and a command to start the test. The server starts the skill test based on this, and the test screen is displayed on the terminal as output.

[0805] Step 5: Perform a skills test

[0806] The user takes the skill test by following the instructions displayed on the screen. The input is each question in the test that is displayed on the screen. The user answers the questions, and the device sends the answers to the server in real time. The output is the answer data sent to the server.

[0807] Step 6: Collecting Biometric Data

[0808] The terminal collects biometric data using cameras, microphones, and sensors. The inputs include data such as the candidate's facial expression, tone of voice, and heart rate. The terminal collects this data and sends it to the server. The output is the biometric data sent to the server.

[0809] Step 7: Analyze the data

[0810] The server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data to evaluate the candidate's stress level and concentration. The inputs are the response data and biometric data. The server analyzes the data using Python scripts and TensorFlow, and the output is the evaluation results.

[0811] Step 8: Generate an overall score

[0812] The server combines the skill assessment and biometric assessment to generate an overall score. The inputs are the results of the skill assessment and biometric assessment. The server combines these assessments and calculates an overall score as output.

[0813] Step 9: Shortlist the best candidates

[0814] The server compares the overall score with the company's requirements and lists the most suitable candidates. The input is the overall score and the company's requirements. The server compares these and generates a list of the most suitable candidates as output.

[0815] Step 10: Providing the best candidates

[0816] The server provides companies with the best candidates. As input, it has the information of the candidates listed. The server provides this to the company and as output, it provides the information in a format that the company can view.

[0817] This process allows the system to eliminate bias and ensure an objective and fair hiring process.

[0818] (Application example 1)

[0819] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0820] Traditional recruitment systems can be biased when evaluating candidates' skills, aptitude, and personality, making it difficult to evaluate them fairly. Furthermore, systems that evaluate the effectiveness of advertising are unable to incorporate real-time responses from viewers, making it difficult to provide personalized advertising. For these reasons, there is a need for more accurate and objective evaluation systems in both the fields of candidate evaluation and ad personalization.

[0821] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0822] In this invention, the server includes means for collecting biometric data to evaluate the skills, aptitude, and personality of candidates, means for the candidate to take a skill test according to instructions displayed on the screen, means for analyzing the skill test results and biometric data in real time to generate an overall score for the candidate, means for compiling a list of optimal candidates based on conditions preset by the company, means for providing the optimal candidates to the company, means for collecting biometric data of advertisement viewers and personalizing advertisements, and means for measuring the effectiveness of advertisements in real time and providing feedback on the results, thereby enabling fair and efficient candidate evaluation and highly accurate advertisement personalization.

[0823] A "candidate" is an individual with specific skills and aptitudes who applies for a role or position sought by a company.

[0824] "Biometric data" is data that digitally records an individual's physiological or behavioral characteristics, such as heart rate, facial expression, and tone of voice.

[0825] A "skills test" is an exam or task designed to assess a candidate's specific knowledge or abilities.

[0826] "Means of real-time analysis" refers to technologies and devices that process data immediately and reflect the results instantly.

[0827] "Composite score" is a number or indicator that indicates the candidate's overall evaluation, generated by integrating the results of the skill test and biometric data.

[0828] "Conditions set in advance by the company" are the standards set in advance by the company regarding the skills, aptitudes, and other requirements that the company requires of job candidates.

[0829] A "means of shortlisting the best candidates" is a method or technique for selecting the most suitable candidates based on set criteria.

[0830] An "advertising viewer" is an individual who views an advertisement and is the person to whom the advertisement is directed.

[0831] "Means for personalizing advertising" are methods and technologies for individually tailoring advertising content based on the viewer's characteristics, interests, and responses.

[0832] "Means for measuring the effectiveness of advertising in real time and providing feedback on the results" refers to a system that instantly analyzes the reactions of viewers of advertisements and reflects the results in advertising strategies.

[0833] This invention is a system that objectively evaluates candidates' skills, aptitude, and personality to realize a fair hiring process. The system includes a server, a terminal, and a user. Furthermore, the system can collect responses from advertisement viewers, personalize advertisements, and measure their effectiveness.

[0834] Server Operation

[0835] The server is responsible for storing the desired skills, aptitudes, and evaluation criteria provided by companies in a database. Based on the conditions set by the company, the server manages the candidates' skill tests and assignments and generates an evaluation profile for each candidate.

[0836] When a candidate begins a skill test, the server receives and analyzes the skill test response data and biometric data in real time. This analysis is primarily performed using software such as OpenCV, TensorFlow, and Keras. This integrates the candidate's skill assessment and biometric assessment to generate an overall score. In addition, the server also collects biometric data of ad viewers, measures the effectiveness of personalized advertising, and provides feedback on the results.

[0837] Device behavior

[0838] The terminal is a device used by candidates and ad viewers, and provides an interface for taking skill tests and viewing ads. The terminal is equipped with a camera, microphone, and various sensors, and these devices are used to collect biometric data.

[0839] During the skill test, the device transmits the candidate's input in real time to a server. The device also collects biometric information such as facial expressions, tone of voice, and heart rate of the ad viewer and transmits this information to the server, which allows for personalized advertising.

[0840] User Actions

[0841] Users are candidates or ad viewers, and when they first access the platform, they create an account by entering the necessary personal information. After logging in, candidates take a skill test provided by the server. During the skill test, users answer questions in order by following the instructions displayed on the screen. Once the skill test is complete, users can check the results and identify their strengths and areas for improvement.

[0842] Ad viewers' responses are collected in real time as they watch the ad, and based on their responses, ad content is personalized and reflected in future viewings.

[0843] Specific examples

[0844] For example, if a company sets "programming ability" and "stress management ability" as required criteria, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers as well as facial expressions and tone of voice, and sends these to the server. The server then scores Candidate A's programming skills and evaluates his or her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's required criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0845] In addition, when an ad viewer is viewing a specific ad, their reaction can be collected and analyzed in real time and used to determine the next ad display. A specific example of a prompt sentence could be entered as follows:

[0846] "We want to develop a system that analyzes users' emotional responses and stress levels in real time, and then delivers personalized advertising content that is optimal for them. We will collect and analyze biometric data such as facial expressions, heart rate, and tone of voice, and use this data to evaluate the effectiveness of advertising. As part of ad personalization, we want to reflect content that the viewer is particularly interested in when displaying ads next time."

[0847] In this way, the present invention improves the fairness of candidate evaluation and enables advertising personalization and effectiveness measurement.

[0848] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0849] Step 1:

[0850] The server stores the desired skill sets, aptitudes, and evaluation criteria provided by companies in a database. This creates a foundation for evaluating candidates based on each company's unique criteria. The input is the requirements data provided by the company, and the output is the evaluation criteria stored in the database.

[0851] Step 2:

[0852] The server registers various skill tests and assignments in a database and generates an individual assessment profile for each candidate. The input is the test question data, and the output is the assessment profile for each candidate. This data is used for subsequent analysis.

[0853] Step 3:

[0854] The candidate starts the skill test using the terminal. The terminal sends the candidate's input to the server in real time. The input is the candidate's answer data, and the output is the server that received the data. The server prepares to analyze the received data immediately.

[0855] Step 4:

[0856] The device uses cameras, microphones, and sensors to collect biometric data (facial expressions, tone of voice, heart rate, etc.) of the candidate and transmits it to a server. The input is the collected biometric data, and the output is the data transmitted to the server.

[0857] Step 5:

[0858] The server analyzes the skill test response data and biometric data in real time. Software such as OpenCV, TensorFlow, and Keras is used for the analysis. The input is the response data and biometric data, and the output is the analysis results. Specific operations include facial expression recognition using models, voice tone analysis, and heart rate variability analysis.

[0859] Step 6:

[0860] The server generates an overall score for the candidate based on the analysis results. The input is the analyzed data, and the output is the overall score, which integrates the candidate's skills and stress assessment.

[0861] Step 7:

[0862] The server compares the overall score with the company's set conditions and lists the most suitable candidates. The input is the overall score and the company's conditions, and the output is a list of the most suitable candidates.

[0863] Step 8:

[0864] The server provides companies with the best candidates. The input is the candidate information listed, and the output is the candidate information sent to the company.

[0865] Step 9:

[0866] The ad viewer watches the ad using a device. The device uses a camera, microphone, and sensors to collect the viewer's responses and send them to a server. The input is biometric data from the time the ad was viewed, and the output is the biometric data sent to the server.

[0867] Step 10:

[0868] The server analyzes the biometric data of the ad viewers in real time to measure the effectiveness of the ad. The input is the viewer's biometric data, and the output is the analysis result of the ad's effectiveness. This analysis includes evaluating emotional responses and measuring stress levels.

[0869] Step 11:

[0870] The server personalizes the ads based on the analysis results and provides the most suitable ads to the viewer. The input is the analysis result of the advertising effectiveness, and the output is personalized advertising content. This result is reflected in subsequent viewings.

[0871] Examples:

[0872] For example, if a viewer shows strong interest in a particular ad, the next ad display can be optimized based on that reaction data. This is achieved using technologies such as "generative AI models and prompt sentences."

[0873] Example prompt sentence:

[0874] "We want to develop a system that analyzes users' emotional responses and stress levels in real time, and then delivers personalized advertising content that is optimal for them. We will collect and analyze biometric data such as facial expressions, heart rate, and tone of voice, and use this data to evaluate the effectiveness of advertising. As part of ad personalization, we want to reflect content that the viewer is particularly interested in when displaying ads next time."

[0875] In this way, a system is built that achieves high accuracy and fairness in both candidate evaluation and ad personalization.

[0876] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0877] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. This system includes a means for incorporating an emotion engine to recognize candidates' emotions from biometric data and reflect them in the evaluation.

[0878] Server Operation

[0879] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. This allows evaluation criteria to be set based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate.

[0880] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the terminal and recognizes the candidate's emotions using an emotion engine. Based on this, the server integrates the skill assessment, biometric assessment, and emotion assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0881] Device behavior

[0882] The terminal is the device used by the candidate and provides the interface for taking the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. At the same time, the terminal uses a camera, microphone, and sensors to collect biometric data, which is then transmitted to the server. This biometric data includes facial expressions, tone of voice, heart rate, etc.

[0883] User Actions

[0884] The user is a candidate and creates an account by entering the required personal information on the platform when first accessing it. After logging in, the user accesses the skill test interface provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user can check the results and understand their strengths and areas for improvement.

[0885] Specific examples

[0886] For example, if a company sets the requirements for a candidate as "programming ability," "stress management ability," and "emotional stability," the system works as follows:

[0887] Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers as well as facial expressions and tone of voice, and sends these to the server. The emotion engine analyzes the collected data, recognizes Candidate A's emotions in real time, and evaluates their stress level and concentration.

[0888] The server scores Candidate A's programming skills and generates an overall score by combining the emotional evaluation from the emotion engine and the biometric evaluation. Based on this overall score, the server compares the candidates with the company's requirements and creates a list of the most suitable candidates. The company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0889] In this way, the present invention eliminates bias and provides an objective and fair hiring process, allowing companies to effectively select top talent.

[0890] The processing flow will be explained below.

[0891] Step 1:

[0892] server

[0893] The desired skill sets, aptitude conditions, and evaluation criteria provided by companies are stored in a database, which allows evaluation criteria to be set based on each company's unique conditions.

[0894] Step 2:

[0895] Terminal

[0896] Activate and verify that cameras, microphones, and sensors are working properly, so they are ready to accurately collect biometric and emotional data.

[0897] Step 3:

[0898] User

[0899] Access the platform, enter the required personal information to create an account, and after logging in, access the skill test interface provided by the server.

[0900] Step 4:

[0901] server

[0902] The system verifies the user's login information and displays the appropriate skills test on the screen. The user is then ready to begin the skills test.

[0903] Step 5:

[0904] Terminal

[0905] As the user starts the skill test, the system provides an input interface and transmits the user's input data to the server in real time. It also collects biometric data using cameras, microphones, and sensors and transmits this data to the server.

[0906] Step 6:

[0907] User

[0908] Follow the on-screen instructions to complete the skills test, answering each task in turn and providing additional information as needed.

[0909] Step 7:

[0910] server

[0911] The system analyzes the skill test response data received in real time and quantifies the candidate's skill level. At the same time, it uses an emotion engine to analyze the biometric data sent from the device and recognize the candidate's emotions. Based on the recognized emotional data, it evaluates the candidate's stress level and concentration level.

[0912] Step 8:

[0913] server

[0914] Skill assessment, emotional assessment, and biometric assessment are combined to generate an overall score for the candidate, which reflects the candidate's skills, aptitudes, emotions, and personality.

[0915] Step 9:

[0916] server

[0917] Based on the generated overall score, the system compares the results with the company's requirements to create a list of the most suitable candidates, and provides information about the candidates to the company.

[0918] Step 10:

[0919] Terminal

[0920] It displays a detailed report of completed skill tests and assessment results to users, allowing them to receive feedback on their skills and aptitudes.

[0921] Step 11:

[0922] company

[0923] The company reviews the list of candidates provided by the server and plans interviews or additional evaluations as the next step. Finally, the company hires the best candidate.

[0924] Example 2

[0925] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0926] In the traditional recruitment process, it is difficult to objectively evaluate a candidate's skills, aptitude, and personality, and the evaluator's subjectivity and bias are easily incorporated, making it difficult to conduct a fair evaluation and select the right talent.In addition, because the evaluation does not take into account the candidate's emotions or stress levels, it is difficult to accurately predict how a candidate will perform in a real work environment.

[0927] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0928] In this invention, the server includes means for storing the skill sets, aptitude conditions, and evaluation criteria provided by the company in a database, means for registering various skill tests and tasks in the database and generating an individual evaluation profile for each candidate, means for receiving and analyzing candidate skill test response data in real time, means for analyzing biometric data and recognizing the candidate's emotions using an emotion engine, means for generating an overall score by integrating the skill evaluation, biometric evaluation, and emotion evaluation, means for comparing the overall score with the company's conditions to list the most suitable candidates, and means for providing the most suitable candidates to the company. This makes it possible to objectively evaluate the skills, aptitude, and personality of candidates and realize a fair hiring process that eliminates bias.

[0929] "Company conditions" refers to the skill sets, aptitudes, and evaluation criteria required by the company.

[0930] "Database" refers to a system for storing and managing data such as information provided by companies, skill tests, assignments, and candidate evaluation profiles.

[0931] A "skills test" refers to a series of questions or tasks administered to assess a candidate's abilities or skills.

[0932] "Biometric data" refers to a candidate's biological information, specifically facial expressions, tone of voice, heart rate, etc.

[0933] An "emotion engine" refers to software or algorithms that analyze collected biometric data and recognize a candidate's emotions.

[0934] "Skills assessment" refers to a method of quantifying the results of a candidate's skills test and evaluating their abilities.

[0935] "Biometric assessment" refers to a method of assessing a candidate's status or characteristics based on collected biometric data.

[0936] "Emotional assessment" refers to an assessment based on the candidate's emotional state recognized using an emotion engine.

[0937] "Composite Score" refers to a candidate's overall assessment score generated by integrating skill assessment, biometric assessment, and emotional assessment.

[0938] The "best candidate" is the candidate who best meets the company's requirements.

[0939] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. The system includes a means for incorporating candidates' emotions into the evaluation by combining an emotion engine with analysis of biometric data.

[0940] Server Operation

[0941] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. Specifically, by registering the information provided by the company in the database, each company's unique criteria are saved. The server also registers skill tests and assignments in the database. Processing is then carried out to generate an individual evaluation profile for each candidate.

[0942] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, the server receives biometric data sent from the device. This biometric data includes facial expressions, tone of voice, heart rate, etc. The server uses an emotion engine to analyze the biometric data and recognize the candidate's emotions.

[0943] The server combines the skill, biometric, and emotional assessments to generate a comprehensive score, which is then matched with the company's requirements to create a list of the most suitable candidates and provide them to the company.

[0944] Device behavior

[0945] The terminal is the device used by the candidate and provides the interface for the skills test. When the candidate starts the skills test, the terminal transmits answer data in real time to the server, including all data entered by the candidate.

[0946] At the same time, the device collects biometric data using cameras, microphones, and sensors, such as a webcam, microphone, and heart rate sensor, and transmits this biometric data to a server to assess the candidate's emotional state.

[0947] User Actions

[0948] The user is a candidate, and when accessing the platform for the first time, they create an account by entering the necessary personal information. After logging in, they access the skill test interface provided by the server. The candidate answers the questions by following the instructions displayed on the screen.

[0949] Once the skills test is complete, users can review their results to identify their strengths and areas for improvement.

[0950] Specific examples

[0951] For example, if a company sets requirements for "programming skills," "stress management skills," and "emotional stability," the system works as follows:

[0952] Candidate A logs into the platform and takes a programming skills test.

[0953] During the test, the device collects Candidate A's answers as well as his / her facial expressions and tone of voice, and sends them to the server.

[0954] The emotion engine analyzes the collected data, recognizes Candidate A's emotions in real time, and assesses his stress level and concentration.

[0955] The server scores Candidate A's programming skills, and generates an overall score by combining the emotional evaluation by the emotion engine and the biometric evaluation. Based on this overall score, the server compares the candidates with the company's requirements and creates a list of the most suitable candidates.

[0956] The company checks the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[0957] In this way, the present invention eliminates bias and provides an objective and fair hiring process, allowing companies to effectively select top talent.

[0958] Example prompts to input to the generative AI model

[0959] System Overview: A system that objectively evaluates candidates' skills, aptitude, and personality to ensure a fair hiring process.

[0960] Technologies used: Emotion engine, biometric data analysis

[0961] Example: A company sets evaluation criteria such as "programming ability," "stress management ability," and "emotional stability."

[0962] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0963] Step 1: Store company information in a database

[0964] The server receives the skill sets, aptitude conditions, and evaluation criteria provided by the companies and stores them in a database.

[0965] Input: Skill set, qualifications, and evaluation criteria provided by the company

[0966] Output: Company condition information stored in the database

[0967] Specific operation: The server acquires skill sets such as "Java programming," "project management," and "communication skills" sent by Company A and stores them in a database called "Company A_Evaluation Criteria."

[0968] Step 2: Register for the Skills Test and Assignment

[0969] The server registers various skill tests and assignments in a database and generates an evaluation profile for each candidate.

[0970] Input: Skill test, assignment information

[0971] Output: Assessment profile for each candidate

[0972] Specific operation: The server registers the "Java Programming Proficiency Test" and "Project Presentation Assignment" in the database and creates a "Candidate A_Evaluation Profile" for Candidate A.

[0973] Step 3: Initiating the skill test and receiving response data

[0974] When a candidate starts a skill test, the device sends answer data to the server in real time.

[0975] Input: Candidate's answer data

[0976] Output: Answer data sent to the server

[0977] Specific operation: Every time the user clicks "Next Question," the device sends the answer data to the server in real time.

[0978] Step 4: Collect and analyze biometric data

[0979] The device uses a camera, microphone, and sensors to collect biometric data and transmits it to a server, which then uses an emotion engine to analyze the biometric data and recognize the candidate's emotions.

[0980] Input: Biometric data (facial expression, tone of voice, heart rate)

[0981] Output: Parsed emotion data

[0982] Specific operation: The device captures facial expressions with a camera, records voice tone with a microphone, and measures heart rate with a heart rate sensor, and sends this data to a server. The emotion engine analyzes the received data and calculates stress levels and concentration levels.

[0983] Step 5: Integrate the data and generate an overall score

[0984] The server combines the skill assessment, biometric assessment, and emotional assessment to generate an overall score.

[0985] Input: Skill assessment data, Biometric assessment data, Emotion assessment data

[0986] Output: Overall score

[0987] Specific operation: The server calculates a skill score (e.g., "85 points / 100 points") based on the answer data, and also performs emotional evaluation using an emotion engine and biometric evaluation, integrating these to generate "Candidate A_Overall Score" (e.g., "Score: 90 / 100").

[0988] Step 6: Matching with companies and providing candidate lists

[0989] The server compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[0990] Input: Overall score, company conditions

[0991] Output: A list of the best candidates

[0992] Specific operation: The server compares "Candidate A_Total Score" with the conditions of Company A, determines from the comparison results that "Candidate A meets the conditions of Company A" and adds him to the list. The company then checks this list and selects the most suitable candidate.

[0993] (Application example 2)

[0994] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0995] In the modern hiring process, it is difficult to fairly evaluate a candidate's skills, aptitude, and personality. Furthermore, these evaluations tend to rely on subjective judgment and are prone to bias. For candidates working in retail stores, evaluating their customer service attitude and stress tolerance is particularly important, but traditional methods make these evaluations difficult. Furthermore, there is a lack of technology that can grasp a candidate's emotional state in real time and reflect that information in the overall evaluation.

[0996] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting biometric data to evaluate the skills, aptitude, and personality of candidates; means for the candidate to take a skill test according to instructions displayed on the screen; means for analyzing the skill test results and biometric data in real time and generating an overall score for the candidate; means for comparing the results with registered evaluation criteria and listing the candidates who best meet the conditions; means for providing the most appropriate candidates to the company; means for analyzing facial expressions and voice in real time using a camera, microphone, and sensor to evaluate the candidate's emotions and stress level; and means for presenting the evaluation results using an overall score based on the customer service attitude and stress tolerance set by the company. This enables a comprehensive evaluation of the candidate's skills and emotional state and a fair and comprehensive judgment.

[0997] A "candidate" is a person who is being evaluated during the recruitment process.

[0998] "Skills" are specific skills or abilities that a candidate possesses.

[0999] "Aptitude" refers to a candidate's ability to fit into a particular job or environment.

[1000] "Personality" refers to the candidate's character and behavioral traits.

[1001] "Biometric data" refers to data based on a candidate's biological information, such as facial expression analysis, voice analysis, and heart rate.

[1002] A "skills test" is an examination to evaluate a candidate's skills and abilities.

[1003] The "total score" is an evaluation index calculated by integrating the results of the skill test and biometric data.

[1004] The "emotion recognition model" is an artificial intelligence model that recognizes candidates' emotions in real time from biometric data.

[1005] "Real-time analysis" is the process of processing data in real time and outputting the results immediately.

[1006] "Evaluation criteria" are standard values ​​used to quantify the skills and aptitudes that companies are looking for.

[1007] "Emotional state" refers to the candidate's current emotional state, including emotions such as joy, anger, sadness, and happiness.

[1008] "Stress tolerance" refers to a candidate's ability to resist and adapt to stress.

[1009] "Customer service attitude" refers to the attitude and behavior displayed by the candidate when performing customer service duties.

[1010] "Company" refers to the organization or entity seeking to recruit a Candidate.

[1011] The present invention provides a system for objectively evaluating a candidate's skills, aptitude, and personality to realize a fair hiring process. Specific embodiments are described below.

[1012] Server Operation

[1013] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. This allows evaluation criteria to be set based on each company's unique conditions. Next, the server registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate.

[1014] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it processes the biometric data sent from the device and recognizes the candidate's emotions using an emotion recognition model. Based on this, the server integrates the skill assessment, biometric assessment, and emotion assessment to generate an overall score. Finally, it compares this overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[1015] Device behavior

[1016] The terminal is the device used by the candidate and provides the interface for taking the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. At the same time, the terminal uses a camera, microphone, and sensors to collect biometric data, which is then transmitted to the server. This biometric data includes facial expressions, tone of voice, heart rate, etc.

[1017] User Actions

[1018] The user is a candidate and creates an account by entering the required personal information on the platform when first accessing it. After logging in, the user accesses the skill test interface provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user can check the results and understand their strengths and areas for improvement.

[1019] Hardware and software used

[1020] The hardware used in implementing this invention includes a smartphone, smart glasses, and a webcam. The software used includes OpenCV, Dlib, Keras (with TensorFlow backend), Python, and emotion recognition models. This combination of hardware and software allows for real-time data collection, analysis, and display.

[1021] Specific examples

[1022] For example, if a candidate is considering a position at a high-end restaurant in Tokyo, the system will evaluate the candidate's smile, nervousness, and friendliness in real time during the interview. In this example, the following prompts are used:

[1023] "Among your friends and family, what are the characteristics of people you find particularly approachable? Please describe specifically what their facial expressions are like."

[1024] The prompts allow candidates to be assessed on their emotional stability and customer service attitude, resulting in an overall score, enabling companies to ensure an objective and fair hiring process and effectively select the best candidates.

[1025] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1026] Step 1:

[1027] When a user first accesses the platform, they create an account by entering the necessary personal information, including basic information such as name, contact details, and resume, and the server receives this data and stores it in a database.

[1028] Step 2:

[1029] The user logs in and accesses the skill test interface provided by the server. Based on the login information, the server generates a skill test interface dedicated to the user and displays it on the screen.

[1030] Step 3:

[1031] The user follows the on-screen instructions to begin the skill test, and each response is sent in real time from the device to the server, which analyzes the response data and monitors the user's progress.

[1032] Step 4:

[1033] The device will begin collecting biometric data about the user using cameras, microphones, and sensors, including facial expressions, tone of voice, and heart rate, in real time, and will send this data to a server.

[1034] Step 5:

[1035] The server inputs the received biometric data into an emotion recognition model to analyze the user's emotional state. The emotion recognition model uses facial expression recognition and voice analysis algorithms to assess the user's current emotions and stress levels.

[1036] Step 6:

[1037] The server combines the skill test results with the emotional assessment obtained from biometric data to generate an overall score. Skill assessment is based on the accuracy rate and speed of answers, while emotional assessment is based on emotional stability and stress tolerance.

[1038] Step 7:

[1039] The server compares the overall score with the evaluation criteria set by the company and lists the candidates who best fit the criteria. This determines how well the candidate's overall evaluation matches the company's requirements.

[1040] Step 8:

[1041] The server provides companies with the best candidates and allows them to view the assessment results on a dashboard, including detailed skill test results, biometric assessment, and overall score.

[1042] Step 9:

[1043] Based on the results of the assessment, companies can decide on interviews or next steps in the selection process. Using the dashboard, company personnel can view detailed assessments of candidates and input further questions and prompts into the generative AI model for further assessment.

[1044] For example, the following prompt text can be used:

[1045] "Among your friends and family, what are the characteristics of people you find particularly approachable? Please describe specifically what their facial expressions are like."

[1046] This process allows companies to comprehensively evaluate a candidate's skills and emotional state, allowing them to make a fair and comprehensive decision.

[1047] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1048] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1049] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1050] [Fourth embodiment]

[1051] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1052] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1053] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1054] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1055] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1056] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1057] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1058] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1059] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1060] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1061] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1062] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1063] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1064] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. In this system, various processes are performed based on the roles of the server, terminal, and user.

[1065] Server Operation

[1066] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by companies in a database. This allows candidates to be evaluated based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate. This creates a foundation for evaluating candidates in a standardized manner.

[1067] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the device to evaluate the candidate's stress level and concentration. Based on this, the server combines the skill assessment and biometric assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[1068] Device behavior

[1069] The terminal is the device used by the candidate and provides the interface for performing the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. The terminal also uses cameras, microphones, and sensors to collect biometric data and transmits the data to the server. This data includes the candidate's facial expressions, tone of voice, heart rate, etc.

[1070] User Actions

[1071] The user is a candidate and creates an account by entering the necessary personal information on the platform when first accessing it. After logging in, the user takes a skill test provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user checks the results and understands their strengths and areas for improvement.

[1072] Specific examples

[1073] For example, if a company sets "programming ability" and "stress management ability" as requirements, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers, as well as facial expressions and tone of voice, and sends them to the server.

[1074] The server scores Candidate A's programming skills and evaluates his / her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's desired criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's criteria.

[1075] In this way, the present invention eliminates bias and ensures an objective and fair hiring process.

[1076] The processing flow will be explained below.

[1077] Step 1:

[1078] server

[1079] The desired skill sets, aptitude conditions, and evaluation criteria provided by companies are stored in a database, which allows evaluation criteria to be set based on each company's unique conditions.

[1080] Step 2:

[1081] Terminal

[1082] Activate and verify that your cameras, microphones, and sensors are working properly, so they are ready to accurately collect biometric data.

[1083] Step 3:

[1084] User

[1085] Access the platform, enter the required personal information to create an account, and after logging in, access the skill test interface provided by the server.

[1086] Step 4:

[1087] server

[1088] The system verifies the user's login information and displays the appropriate skills test on the screen. The user is then ready to begin the skills test.

[1089] Step 5:

[1090] Terminal

[1091] As the user starts the skill test, the system provides an input interface, transmits the user's input data to the server in real time, and collects the candidate's biometric data, which is also transmitted to the server.

[1092] Step 6:

[1093] User

[1094] Follow the on-screen instructions to complete the skills test, answering each task in turn and providing additional information as needed.

[1095] Step 7:

[1096] server

[1097] The system analyzes the skill test response data received in real time to quantify the candidate's skill level, while analyzing the biometric data sent from the device to evaluate the candidate's stress level and concentration.

[1098] Step 8:

[1099] server

[1100] Skill assessment and biometric assessment are combined to generate an overall score for the candidate, which reflects the candidate's skills, aptitudes, and personality.

[1101] Step 9:

[1102] server

[1103] Based on the generated overall score, the system compares the results with the company's requirements to create a list of the most suitable candidates, and provides information about the candidates to the company.

[1104] Step 10:

[1105] Terminal

[1106] It displays a detailed report of completed skill tests and assessment results to users, allowing them to receive feedback on their skills and aptitudes.

[1107] Step 11:

[1108] company

[1109] The company reviews the list of candidates provided by the server and plans interviews or additional evaluations as the next step. Finally, the company hires the best candidate.

[1110] Example 1

[1111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1112] Traditional hiring processes are prone to subjective evaluation and bias, making it difficult to conduct fair evaluations. It's also difficult to accurately assess applicants' skills and aptitude, making it difficult to hire the best candidates. Furthermore, important factors like a candidate's stress level and level of concentration tend to be overlooked, resulting in cases where candidates don't accurately match the talent specifications companies are looking for.

[1113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1114] In this invention, the server includes a means for storing the skill sets, aptitude criteria, and evaluation criteria provided by the company in a database, a means for analyzing the skill test results and biometric data in real time to generate an overall score for the candidate, and a means for integrating the skill evaluation and biometric evaluation to generate an overall score. This eliminates subjective bias and enables the evaluation of candidates in an objective and fair manner. Factors such as stress level and concentration are also taken into account, making it possible to list the most suitable candidates who meet the company's desired criteria.

[1115] "Biometric data" refers to data that measures a candidate's physical characteristics and is stored digitally, such as facial expressions, tone of voice, and heart rate.

[1116] A "skills test" is a test administered to assess a candidate's specific abilities or knowledge. Examples include programming tests and tasks that assess problem-solving ability.

[1117] The "total score" is an evaluation of a candidate calculated based on the results of the skill test and biometric data. It is used to comprehensively judge the candidate's abilities and aptitude.

[1118] The "database" is an information management system for storing the skill sets, aptitude conditions, and evaluation criteria provided by the company. Specifically, a database management system such as MySQL is used.

[1119] "Real-time analysis" is the process of analyzing skill test results and biometric data as they are collected, allowing for rapid candidate evaluation.

[1120] "A means of listing the best candidates based on a company's requirements" is a method for selecting candidates with a high overall score based on evaluation criteria set by the company in advance. This method allows companies to efficiently identify the talent they are looking for.

[1121] "Cameras, microphones, and sensors" are hardware devices used to collect biometric data on candidates, including facial expressions, tone of voice, and heart rate.

[1122] "Means for quantifying skill levels" refers to a method for expressing a candidate's specific abilities as a number based on the results of a skill test. This allows for an objective evaluation of the candidate's abilities.

[1123] This invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. This system performs various processes based on the roles of the server, terminal, and user.

[1124] Server Operation

[1125] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by companies in a database. This allows candidates to be evaluated based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate. This creates a foundation for evaluating candidates in a standardized manner.

[1126] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the device to evaluate the candidate's stress level and concentration. Based on this, the server combines the skill assessment and biometric assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[1127] Specific hardware and software features include a server running Python and a MySQL database for data processing and analysis, and TensorFlow and Keras machine learning algorithms for analyzing biometric data.

[1128] Device behavior

[1129] The terminal is the device used by the candidate and provides the interface for performing the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. The terminal also uses cameras, microphones, and sensors to collect biometric data and transmits the data to the server. This data includes the candidate's facial expressions, tone of voice, heart rate, etc.

[1130] Specific hardware such as cameras, microphones, and sensors are required to accurately collect biometric data, which can be used to assess a candidate's stress level, concentration, etc. in real time.

[1131] User Actions

[1132] The user is a candidate and creates an account by entering the necessary personal information on the platform when first accessing it. After logging in, the user takes a skill test provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user checks the results and understands their strengths and areas for improvement.

[1133] Specific examples

[1134] For example, if a company sets "programming ability" and "stress management ability" as requirements, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers, as well as facial expressions and tone of voice, and sends them to the server.

[1135] The server scores Candidate A's programming skills and evaluates his / her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's desired criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's criteria.

[1136] Prompt Sentence Examples

[1137] An example prompt is:

[1138] "What methods and techniques will the system use to assess candidates' programming and stress management abilities?"

[1139] In this way, the present invention eliminates bias and ensures an objective and fair hiring process.

[1140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1141] Step 1: Register your company information

[1142] The server stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. The input is information provided by the company (e.g., programming ability, teamwork ability). The server stores this information in the database, and the company's conditions are registered in the database as output.

[1143] Step 2: Register for the skills test

[1144] The server registers the skill tests and assignments provided by the company in a database. The input is the skill tests and assignments (e.g. Java programming tests) provided by the company. The server saves these in the database, and the registered skill tests and assignments are stored in the database as output.

[1145] Step 3: Create a candidate account

[1146] A user creates an account by entering the necessary personal information when first accessing the site. The input includes the personal information provided by the user (e.g., name, email address, password). The server stores this information in a database, and the output is a user account.

[1147] Step 4: Start the skills test

[1148] After logging in, the user starts the skill test provided by the server. The input is the user's login information and a command to start the test. The server starts the skill test based on this, and the test screen is displayed on the terminal as output.

[1149] Step 5: Perform a skills test

[1150] The user takes the skill test by following the instructions displayed on the screen. The input is each question in the test that is displayed on the screen. The user answers the questions, and the device sends the answers to the server in real time. The output is the answer data sent to the server.

[1151] Step 6: Collecting Biometric Data

[1152] The terminal collects biometric data using cameras, microphones, and sensors. The inputs include data such as the candidate's facial expression, tone of voice, and heart rate. The terminal collects this data and sends it to the server. The output is the biometric data sent to the server.

[1153] Step 7: Analyze the data

[1154] The server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data to evaluate the candidate's stress level and concentration. The inputs are the response data and biometric data. The server analyzes the data using Python scripts and TensorFlow, and the output is the evaluation results.

[1155] Step 8: Generate an overall score

[1156] The server combines the skill assessment and biometric assessment to generate an overall score. The inputs are the results of the skill assessment and biometric assessment. The server combines these assessments and calculates an overall score as output.

[1157] Step 9: Shortlist the best candidates

[1158] The server compares the overall score with the company's requirements and lists the most suitable candidates. The input is the overall score and the company's requirements. The server compares these and generates a list of the most suitable candidates as output.

[1159] Step 10: Providing the best candidates

[1160] The server provides companies with the best candidates. As input, it has the information of the candidates listed. The server provides this to the company and as output, it provides the information in a format that the company can view.

[1161] This process allows the system to eliminate bias and ensure an objective and fair hiring process.

[1162] (Application example 1)

[1163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1164] Traditional recruitment systems can be biased when evaluating candidates' skills, aptitude, and personality, making it difficult to evaluate them fairly. Furthermore, systems that evaluate the effectiveness of advertising are unable to incorporate real-time responses from viewers, making it difficult to provide personalized advertising. For these reasons, there is a need for more accurate and objective evaluation systems in both the fields of candidate evaluation and ad personalization.

[1165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1166] In this invention, the server includes means for collecting biometric data to evaluate the skills, aptitude, and personality of candidates, means for the candidate to take a skill test according to instructions displayed on the screen, means for analyzing the skill test results and biometric data in real time to generate an overall score for the candidate, means for compiling a list of optimal candidates based on conditions preset by the company, means for providing the optimal candidates to the company, means for collecting biometric data of advertisement viewers and personalizing advertisements, and means for measuring the effectiveness of advertisements in real time and providing feedback on the results, thereby enabling fair and efficient candidate evaluation and highly accurate advertisement personalization.

[1167] A "candidate" is an individual with specific skills and aptitudes who applies for a role or position sought by a company.

[1168] "Biometric data" is data that digitally records an individual's physiological or behavioral characteristics, such as heart rate, facial expression, and tone of voice.

[1169] A "skills test" is an exam or task designed to assess a candidate's specific knowledge or abilities.

[1170] "Means of real-time analysis" refers to technologies and devices that process data immediately and reflect the results instantly.

[1171] "Composite score" is a number or indicator that indicates the candidate's overall evaluation, generated by integrating the results of the skill test and biometric data.

[1172] "Conditions set in advance by the company" are the standards set in advance by the company regarding the skills, aptitudes, and other requirements that the company requires of job candidates.

[1173] A "means of shortlisting the best candidates" is a method or technique for selecting the most suitable candidates based on set criteria.

[1174] An "advertising viewer" is an individual who views an advertisement and is the person to whom the advertisement is directed.

[1175] "Means for personalizing advertising" are methods and technologies for individually tailoring advertising content based on the viewer's characteristics, interests, and responses.

[1176] "Means for measuring the effectiveness of advertising in real time and providing feedback on the results" refers to a system that instantly analyzes the reactions of viewers of advertisements and reflects the results in advertising strategies.

[1177] This invention is a system that objectively evaluates candidates' skills, aptitude, and personality to realize a fair hiring process. The system includes a server, a terminal, and a user. Furthermore, the system can collect responses from advertisement viewers, personalize advertisements, and measure their effectiveness.

[1178] Server Operation

[1179] The server is responsible for storing the desired skills, aptitudes, and evaluation criteria provided by companies in a database. Based on the conditions set by the company, the server manages the candidates' skill tests and assignments and generates an evaluation profile for each candidate.

[1180] When a candidate begins a skill test, the server receives and analyzes the skill test response data and biometric data in real time. This analysis is primarily performed using software such as OpenCV, TensorFlow, and Keras. This integrates the candidate's skill assessment and biometric assessment to generate an overall score. In addition, the server also collects biometric data of ad viewers, measures the effectiveness of personalized advertising, and provides feedback on the results.

[1181] Device behavior

[1182] The terminal is a device used by candidates and ad viewers, and provides an interface for taking skill tests and viewing ads. The terminal is equipped with a camera, microphone, and various sensors, and these devices are used to collect biometric data.

[1183] During the skill test, the device transmits the candidate's input in real time to a server. The device also collects biometric information such as facial expressions, tone of voice, and heart rate of the ad viewer and transmits this information to the server, which allows for personalized advertising.

[1184] User Actions

[1185] Users are candidates or ad viewers, and when they first access the platform, they create an account by entering the necessary personal information. After logging in, candidates take a skill test provided by the server. During the skill test, users answer questions in order by following the instructions displayed on the screen. Once the skill test is complete, users can check the results and identify their strengths and areas for improvement.

[1186] Ad viewers' responses are collected in real time as they watch the ad, and based on their responses, ad content is personalized and reflected in future viewings.

[1187] Specific examples

[1188] For example, if a company sets "programming ability" and "stress management ability" as required criteria, the system works as follows: Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers as well as facial expressions and tone of voice, and sends these to the server. The server then scores Candidate A's programming skills and evaluates his or her stress management ability based on biometric data. These evaluations are combined to generate an overall score, which is then compared with the company's required criteria. Finally, the company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[1189] In addition, when an ad viewer is viewing a specific ad, their reaction can be collected and analyzed in real time and used to determine the next ad display. A specific example of a prompt sentence could be entered as follows:

[1190] "We want to develop a system that analyzes users' emotional responses and stress levels in real time, and then delivers personalized advertising content that is optimal for them. We will collect and analyze biometric data such as facial expressions, heart rate, and tone of voice, and use this data to evaluate the effectiveness of advertising. As part of ad personalization, we want to reflect content that the viewer is particularly interested in when displaying ads next time."

[1191] In this way, the present invention improves the fairness of candidate evaluation and enables advertising personalization and effectiveness measurement.

[1192] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1193] Step 1:

[1194] The server stores the desired skill sets, aptitudes, and evaluation criteria provided by companies in a database. This creates a foundation for evaluating candidates based on each company's unique criteria. The input is the requirements data provided by the company, and the output is the evaluation criteria stored in the database.

[1195] Step 2:

[1196] The server registers various skill tests and assignments in a database and generates an individual assessment profile for each candidate. The input is the test question data, and the output is the assessment profile for each candidate. This data is used for subsequent analysis.

[1197] Step 3:

[1198] The candidate starts the skill test using the terminal. The terminal sends the candidate's input to the server in real time. The input is the candidate's answer data, and the output is the server that received the data. The server prepares to analyze the received data immediately.

[1199] Step 4:

[1200] The device uses cameras, microphones, and sensors to collect biometric data (facial expressions, tone of voice, heart rate, etc.) of the candidate and transmits it to a server. The input is the collected biometric data, and the output is the data transmitted to the server.

[1201] Step 5:

[1202] The server analyzes the skill test response data and biometric data in real time. Software such as OpenCV, TensorFlow, and Keras is used for the analysis. The input is the response data and biometric data, and the output is the analysis results. Specific operations include facial expression recognition using models, voice tone analysis, and heart rate variability analysis.

[1203] Step 6:

[1204] The server generates an overall score for the candidate based on the analysis results. The input is the analyzed data, and the output is the overall score, which integrates the candidate's skills and stress assessment.

[1205] Step 7:

[1206] The server compares the overall score with the company's set conditions and lists the most suitable candidates. The input is the overall score and the company's conditions, and the output is a list of the most suitable candidates.

[1207] Step 8:

[1208] The server provides companies with the best candidates. The input is the candidate information listed, and the output is the candidate information sent to the company.

[1209] Step 9:

[1210] The ad viewer watches the ad using a device. The device uses a camera, microphone, and sensors to collect the viewer's responses and send them to a server. The input is biometric data from the time the ad was viewed, and the output is the biometric data sent to the server.

[1211] Step 10:

[1212] The server analyzes the biometric data of the ad viewers in real time to measure the effectiveness of the ad. The input is the viewer's biometric data, and the output is the analysis result of the ad's effectiveness. This analysis includes evaluating emotional responses and measuring stress levels.

[1213] Step 11:

[1214] The server personalizes the ads based on the analysis results and provides the most suitable ads to the viewer. The input is the analysis result of the advertising effectiveness, and the output is personalized advertising content. This result is reflected in subsequent viewings.

[1215] Examples:

[1216] For example, if a viewer shows strong interest in a particular ad, the next ad display can be optimized based on that reaction data. This is achieved using technologies such as "generative AI models and prompt sentences."

[1217] Example prompt sentence:

[1218] "We want to develop a system that analyzes users' emotional responses and stress levels in real time, and then delivers personalized advertising content that is optimal for them. We will collect and analyze biometric data such as facial expressions, heart rate, and tone of voice, and use this data to evaluate the effectiveness of advertising. As part of ad personalization, we want to reflect content that the viewer is particularly interested in when displaying ads next time."

[1219] In this way, a system is built that achieves high accuracy and fairness in both candidate evaluation and ad personalization.

[1220] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1221] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. This system includes a means for incorporating an emotion engine to recognize candidates' emotions from biometric data and reflect them in the evaluation.

[1222] Server Operation

[1223] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. This allows evaluation criteria to be set based on each company's unique conditions. The server also registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate.

[1224] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it analyzes the biometric data sent from the terminal and recognizes the candidate's emotions using an emotion engine. Based on this, the server integrates the skill assessment, biometric assessment, and emotion assessment to generate an overall score. Finally, it compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[1225] Device behavior

[1226] The terminal is the device used by the candidate and provides the interface for taking the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. At the same time, the terminal uses a camera, microphone, and sensors to collect biometric data, which is then transmitted to the server. This biometric data includes facial expressions, tone of voice, heart rate, etc.

[1227] User Actions

[1228] The user is a candidate and creates an account by entering the required personal information on the platform when first accessing it. After logging in, the user accesses the skill test interface provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user can check the results and understand their strengths and areas for improvement.

[1229] Specific examples

[1230] For example, if a company sets the requirements for a candidate as "programming ability," "stress management ability," and "emotional stability," the system works as follows:

[1231] Candidate A logs in to the platform and takes a programming skills test. During the test, the device collects Candidate A's answers as well as facial expressions and tone of voice, and sends these to the server. The emotion engine analyzes the collected data, recognizes Candidate A's emotions in real time, and evaluates their stress level and concentration.

[1232] The server scores Candidate A's programming skills and generates an overall score by combining the emotional evaluation from the emotion engine and the biometric evaluation. Based on this overall score, the server compares the candidates with the company's requirements and creates a list of the most suitable candidates. The company reviews the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[1233] In this way, the present invention eliminates bias and provides an objective and fair hiring process, allowing companies to effectively select top talent.

[1234] The processing flow will be explained below.

[1235] Step 1:

[1236] server

[1237] The desired skill sets, aptitude conditions, and evaluation criteria provided by companies are stored in a database, which allows evaluation criteria to be set based on each company's unique conditions.

[1238] Step 2:

[1239] Terminal

[1240] Activate and verify that cameras, microphones, and sensors are working properly, so they are ready to accurately collect biometric and emotional data.

[1241] Step 3:

[1242] User

[1243] Access the platform, enter the required personal information to create an account, and after logging in, access the skill test interface provided by the server.

[1244] Step 4:

[1245] server

[1246] The system verifies the user's login information and displays the appropriate skills test on the screen. The user is then ready to begin the skills test.

[1247] Step 5:

[1248] Terminal

[1249] As the user starts the skill test, the system provides an input interface and transmits the user's input data to the server in real time. It also collects biometric data using cameras, microphones, and sensors and transmits this data to the server.

[1250] Step 6:

[1251] User

[1252] Follow the on-screen instructions to complete the skills test, answering each task in turn and providing additional information as needed.

[1253] Step 7:

[1254] server

[1255] The system analyzes the skill test response data received in real time and quantifies the candidate's skill level. At the same time, it uses an emotion engine to analyze the biometric data sent from the device and recognize the candidate's emotions. Based on the recognized emotional data, it evaluates the candidate's stress level and concentration level.

[1256] Step 8:

[1257] server

[1258] Skill assessment, emotional assessment, and biometric assessment are combined to generate an overall score for the candidate, which reflects the candidate's skills, aptitudes, emotions, and personality.

[1259] Step 9:

[1260] server

[1261] Based on the generated overall score, the system compares the results with the company's requirements to create a list of the most suitable candidates, and provides information about the candidates to the company.

[1262] Step 10:

[1263] Terminal

[1264] It displays a detailed report of completed skill tests and assessment results to users, allowing them to receive feedback on their skills and aptitudes.

[1265] Step 11:

[1266] company

[1267] The company reviews the list of candidates provided by the server and plans interviews or additional evaluations as the next step. Finally, the company hires the best candidate.

[1268] Example 2

[1269] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1270] In the traditional recruitment process, it is difficult to objectively evaluate a candidate's skills, aptitude, and personality, and the evaluator's subjectivity and bias are easily incorporated, making it difficult to conduct a fair evaluation and select the right talent.In addition, because the evaluation does not take into account the candidate's emotions or stress levels, it is difficult to accurately predict how a candidate will perform in a real work environment.

[1271] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1272] In this invention, the server includes means for storing the skill sets, aptitude conditions, and evaluation criteria provided by the company in a database, means for registering various skill tests and tasks in the database and generating an individual evaluation profile for each candidate, means for receiving and analyzing candidate skill test response data in real time, means for analyzing biometric data and recognizing the candidate's emotions using an emotion engine, means for generating an overall score by integrating the skill evaluation, biometric evaluation, and emotion evaluation, means for comparing the overall score with the company's conditions to list the most suitable candidates, and means for providing the most suitable candidates to the company. This makes it possible to objectively evaluate the skills, aptitude, and personality of candidates and realize a fair hiring process that eliminates bias.

[1273] "Company conditions" refers to the skill sets, aptitudes, and evaluation criteria required by the company.

[1274] "Database" refers to a system for storing and managing data such as information provided by companies, skill tests, assignments, and candidate evaluation profiles.

[1275] A "skills test" refers to a series of questions or tasks administered to assess a candidate's abilities or skills.

[1276] "Biometric data" refers to a candidate's biological information, specifically facial expressions, tone of voice, heart rate, etc.

[1277] An "emotion engine" refers to software or algorithms that analyze collected biometric data and recognize a candidate's emotions.

[1278] "Skills assessment" refers to a method of quantifying the results of a candidate's skills test and evaluating their abilities.

[1279] "Biometric assessment" refers to a method of assessing a candidate's status or characteristics based on collected biometric data.

[1280] "Emotional assessment" refers to an assessment based on the candidate's emotional state recognized using an emotion engine.

[1281] "Composite Score" refers to a candidate's overall assessment score generated by integrating skill assessment, biometric assessment, and emotional assessment.

[1282] The "best candidate" is the candidate who best meets the company's requirements.

[1283] The present invention is a system for objectively evaluating candidates' skills, aptitude, and personality to realize a fair hiring process. The system includes a means for incorporating candidates' emotions into the evaluation by combining an emotion engine with analysis of biometric data.

[1284] Server Operation

[1285] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. Specifically, by registering the information provided by the company in the database, each company's unique criteria are saved. The server also registers skill tests and assignments in the database. Processing is then carried out to generate an individual evaluation profile for each candidate.

[1286] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, the server receives biometric data sent from the device. This biometric data includes facial expressions, tone of voice, heart rate, etc. The server uses an emotion engine to analyze the biometric data and recognize the candidate's emotions.

[1287] The server combines the skill, biometric, and emotional assessments to generate a comprehensive score, which is then matched with the company's requirements to create a list of the most suitable candidates and provide them to the company.

[1288] Device behavior

[1289] The terminal is the device used by the candidate and provides the interface for the skills test. When the candidate starts the skills test, the terminal transmits answer data in real time to the server, including all data entered by the candidate.

[1290] At the same time, the device collects biometric data using cameras, microphones, and sensors, such as a webcam, microphone, and heart rate sensor, and transmits this biometric data to a server to assess the candidate's emotional state.

[1291] User Actions

[1292] The user is a candidate, and when accessing the platform for the first time, they create an account by entering the necessary personal information. After logging in, they access the skill test interface provided by the server. The candidate answers the questions by following the instructions displayed on the screen.

[1293] Once the skills test is complete, users can review their results to identify their strengths and areas for improvement.

[1294] Specific examples

[1295] For example, if a company sets requirements for "programming skills," "stress management skills," and "emotional stability," the system works as follows:

[1296] Candidate A logs into the platform and takes a programming skills test.

[1297] During the test, the device collects Candidate A's answers as well as his / her facial expressions and tone of voice, and sends them to the server.

[1298] The emotion engine analyzes the collected data, recognizes Candidate A's emotions in real time, and assesses his stress level and concentration.

[1299] The server scores Candidate A's programming skills, and generates an overall score by combining the emotional evaluation by the emotion engine and the biometric evaluation. Based on this overall score, the server compares the candidates with the company's requirements and creates a list of the most suitable candidates.

[1300] The company checks the evaluation results provided by the server and determines that Candidate A is the best fit for the company's requirements.

[1301] In this way, the present invention eliminates bias and provides an objective and fair hiring process, allowing companies to effectively select top talent.

[1302] Example prompts to input to the generative AI model

[1303] System Overview: A system that objectively evaluates candidates' skills, aptitude, and personality to ensure a fair hiring process.

[1304] Technologies used: Emotion engine, biometric data analysis

[1305] Example: A company sets evaluation criteria such as "programming ability," "stress management ability," and "emotional stability."

[1306] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1307] Step 1: Store company information in a database

[1308] The server receives the skill sets, aptitude conditions, and evaluation criteria provided by the companies and stores them in a database.

[1309] Input: Skill set, qualifications, and evaluation criteria provided by the company

[1310] Output: Company condition information stored in the database

[1311] Specific operation: The server acquires skill sets such as "Java programming," "project management," and "communication skills" sent by Company A and stores them in a database called "Company A_Evaluation Criteria."

[1312] Step 2: Register for the Skills Test and Assignment

[1313] The server registers various skill tests and assignments in a database and generates an evaluation profile for each candidate.

[1314] Input: Skill test, assignment information

[1315] Output: Assessment profile for each candidate

[1316] Specific operation: The server registers the "Java Programming Proficiency Test" and "Project Presentation Assignment" in the database and creates a "Candidate A_Evaluation Profile" for Candidate A.

[1317] Step 3: Initiating the skill test and receiving response data

[1318] When a candidate starts a skill test, the device sends answer data to the server in real time.

[1319] Input: Candidate's answer data

[1320] Output: Answer data sent to the server

[1321] Specific operation: Every time the user clicks "Next Question," the device sends the answer data to the server in real time.

[1322] Step 4: Collect and analyze biometric data

[1323] The device uses a camera, microphone, and sensors to collect biometric data and transmits it to a server, which then uses an emotion engine to analyze the biometric data and recognize the candidate's emotions.

[1324] Input: Biometric data (facial expression, tone of voice, heart rate)

[1325] Output: Parsed emotion data

[1326] Specific operation: The device captures facial expressions with a camera, records voice tone with a microphone, and measures heart rate with a heart rate sensor, and sends this data to a server. The emotion engine analyzes the received data and calculates stress levels and concentration levels.

[1327] Step 5: Integrate the data and generate an overall score

[1328] The server combines the skill assessment, biometric assessment, and emotional assessment to generate an overall score.

[1329] Input: Skill assessment data, Biometric assessment data, Emotion assessment data

[1330] Output: Overall score

[1331] Specific operation: The server calculates a skill score (e.g., "85 points / 100 points") based on the answer data, and also performs emotional evaluation using an emotion engine and biometric evaluation, integrating these to generate "Candidate A_Overall Score" (e.g., "Score: 90 / 100").

[1332] Step 6: Matching with companies and providing candidate lists

[1333] The server compares the overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[1334] Input: Overall score, company conditions

[1335] Output: A list of the best candidates

[1336] Specific operation: The server compares "Candidate A_Total Score" with the conditions of Company A, determines from the comparison results that "Candidate A meets the conditions of Company A" and adds him to the list. The company then checks this list and selects the most suitable candidate.

[1337] (Application example 2)

[1338] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1339] In the modern hiring process, it is difficult to fairly evaluate a candidate's skills, aptitude, and personality. Furthermore, these evaluations tend to rely on subjective judgment and are prone to bias. For candidates working in retail stores, evaluating their customer service attitude and stress tolerance is particularly important, but traditional methods make these evaluations difficult. Furthermore, there is a lack of technology that can grasp a candidate's emotional state in real time and reflect that information in the overall evaluation.

[1340] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting biometric data to evaluate the skills, aptitude, and personality of candidates; means for the candidate to take a skill test according to instructions displayed on the screen; means for analyzing the skill test results and biometric data in real time and generating an overall score for the candidate; means for comparing the results with registered evaluation criteria and listing the candidates who best meet the conditions; means for providing the most appropriate candidates to the company; means for analyzing facial expressions and voice in real time using a camera, microphone, and sensor to evaluate the candidate's emotions and stress level; and means for presenting the evaluation results using an overall score based on the customer service attitude and stress tolerance set by the company. This enables a comprehensive evaluation of the candidate's skills and emotional state and a fair and comprehensive judgment.

[1341] A "candidate" is a person who is being evaluated during the recruitment process.

[1342] "Skills" are specific skills or abilities that a candidate possesses.

[1343] "Aptitude" refers to a candidate's ability to fit into a particular job or environment.

[1344] "Personality" refers to the candidate's character and behavioral traits.

[1345] "Biometric data" refers to data based on a candidate's biological information, such as facial expression analysis, voice analysis, and heart rate.

[1346] A "skills test" is an examination to evaluate a candidate's skills and abilities.

[1347] The "total score" is an evaluation index calculated by integrating the results of the skill test and biometric data.

[1348] The "emotion recognition model" is an artificial intelligence model that recognizes candidates' emotions in real time from biometric data.

[1349] "Real-time analysis" is the process of processing data in real time and outputting the results immediately.

[1350] "Evaluation criteria" are standard values ​​used to quantify the skills and aptitudes that companies are looking for.

[1351] "Emotional state" refers to the candidate's current emotional state, including emotions such as joy, anger, sadness, and happiness.

[1352] "Stress tolerance" refers to a candidate's ability to resist and adapt to stress.

[1353] "Customer service attitude" refers to the attitude and behavior displayed by the candidate when performing customer service duties.

[1354] "Company" refers to the organization or entity seeking to recruit a Candidate.

[1355] The present invention provides a system for objectively evaluating a candidate's skills, aptitude, and personality to realize a fair hiring process. Specific embodiments are described below.

[1356] Server Operation

[1357] The server first stores the desired skill sets, aptitude conditions, and evaluation criteria provided by the company in a database. This allows evaluation criteria to be set based on each company's unique conditions. Next, the server registers various skill tests and tasks in the database and generates an individual evaluation profile for each candidate.

[1358] When a candidate logs in and begins the skill test, the server receives and analyzes the skill test response data in real time. At the same time, it processes the biometric data sent from the device and recognizes the candidate's emotions using an emotion recognition model. Based on this, the server integrates the skill assessment, biometric assessment, and emotion assessment to generate an overall score. Finally, it compares this overall score with the company's requirements and provides a list of the most suitable candidates to the company.

[1359] Device behavior

[1360] The terminal is the device used by the candidate and provides the interface for taking the skills test. When the candidate starts the skills test, the terminal transmits the user's input to the server in real time. At the same time, the terminal uses a camera, microphone, and sensors to collect biometric data, which is then transmitted to the server. This biometric data includes facial expressions, tone of voice, heart rate, etc.

[1361] User Actions

[1362] The user is a candidate and creates an account by entering the required personal information on the platform when first accessing it. After logging in, the user accesses the skill test interface provided by the server. During the skill test, the user answers the questions in order according to the instructions displayed on the screen. After completing the skill test, the user can check the results and understand their strengths and areas for improvement.

[1363] Hardware and software used

[1364] The hardware used in implementing this invention includes a smartphone, smart glasses, and a webcam. The software used includes OpenCV, Dlib, Keras (with TensorFlow backend), Python, and emotion recognition models. This combination of hardware and software allows for real-time data collection, analysis, and display.

[1365] Specific examples

[1366] For example, if a candidate is considering a position at a high-end restaurant in Tokyo, the system will evaluate the candidate's smile, nervousness, and friendliness in real time during the interview. In this example, the following prompts are used:

[1367] "Among your friends and family, what are the characteristics of people you find particularly approachable? Please describe specifically what their facial expressions are like."

[1368] The prompts allow candidates to be assessed on their emotional stability and customer service attitude, resulting in an overall score, enabling companies to ensure an objective and fair hiring process and effectively select the best candidates.

[1369] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1370] Step 1:

[1371] When a user first accesses the platform, they create an account by entering the necessary personal information, including basic information such as name, contact details, and resume, and the server receives this data and stores it in a database.

[1372] Step 2:

[1373] The user logs in and accesses the skill test interface provided by the server. Based on the login information, the server generates a skill test interface dedicated to the user and displays it on the screen.

[1374] Step 3:

[1375] The user follows the on-screen instructions to begin the skill test, and each response is sent in real time from the device to the server, which analyzes the response data and monitors the user's progress.

[1376] Step 4:

[1377] The device will begin collecting biometric data about the user using cameras, microphones, and sensors, including facial expressions, tone of voice, and heart rate, in real time, and will send this data to a server.

[1378] Step 5:

[1379] The server inputs the received biometric data into an emotion recognition model to analyze the user's emotional state. The emotion recognition model uses facial expression recognition and voice analysis algorithms to assess the user's current emotions and stress levels.

[1380] Step 6:

[1381] The server combines the skill test results with the emotional assessment obtained from biometric data to generate an overall score. Skill assessment is based on the accuracy rate and speed of answers, while emotional assessment is based on emotional stability and stress tolerance.

[1382] Step 7:

[1383] The server compares the overall score with the evaluation criteria set by the company and lists the candidates who best fit the criteria. This determines how well the candidate's overall evaluation matches the company's requirements.

[1384] Step 8:

[1385] The server provides companies with the best candidates and allows them to view the assessment results on a dashboard, including detailed skill test results, biometric assessment, and overall score.

[1386] Step 9:

[1387] Based on the results of the assessment, companies can decide on interviews or next steps in the selection process. Using the dashboard, company personnel can view detailed assessments of candidates and input further questions and prompts into the generative AI model for further assessment.

[1388] For example, the following prompt text can be used:

[1389] "Among your friends and family, what are the characteristics of people you find particularly approachable? Please describe specifically what their facial expressions are like."

[1390] This process allows companies to comprehensively evaluate a candidate's skills and emotional state, allowing them to make a fair and comprehensive decision.

[1391] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1392] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1393] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1394] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1395] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1396] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1397] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1398] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1399] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1400] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1401] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1402] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1403] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1404] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1405] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1406] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1407] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1408] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1409] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1410] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1411] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1412] The following is further disclosed regarding the above embodiment.

[1413] (Claim 1)

[1414] The means of collecting biometric data to assess candidates' skills, aptitudes and personality;

[1415] a means for the candidate to complete the skills test by following on-screen instructions;

[1416] a means for analyzing the skills test results and biometric data in real time to generate an overall score for the candidate; and

[1417] A way for companies to create a list of the best candidates based on pre-defined criteria,

[1418] A system that includes a means of providing companies with the best candidates.

[1419] (Claim 2)

[1420] 10. The system of claim 1, further comprising means for using a camera, a microphone, and a sensor to collect biometric data of the candidate.

[1421] (Claim 3)

[1422] 10. The system of claim 1, further comprising means for quantifying the candidate's skill level based on the results of the skill test.

[1423] "Example 1"

[1424] (Claim 1)

[1425] The means of collecting biometric data to assess candidates' skills, aptitudes and personality;

[1426] a means for the candidate to complete the skills test by following on-screen instructions;

[1427] a means for analyzing the skills test results and biometric data in real time to generate an overall score for the candidate; and

[1428] A way for companies to create a list of the best candidates based on pre-defined criteria,

[1429] A means to provide companies with the best candidates,

[1430] A means to store the skill sets, aptitude conditions, and evaluation criteria provided by companies in a database,

[1431] A system including a means for integrating the skill assessment and the biometric assessment to generate an overall score.

[1432] (Claim 2)

[1433] 10. The system of claim 1, further comprising means for using a camera, a microphone, and a sensor to collect biometric data of the candidate.

[1434] (Claim 3)

[1435] 2. The system of claim 1, further comprising: means for quantifying the candidate's skill level based on the results of the skill test; and means for analyzing the biometric data to assess the candidate's stress level and concentration level.

[1436] "Application Example 1"

[1437] (Claim 1)

[1438] The means of collecting biometric data to assess candidates' skills, aptitudes and personality;

[1439] a means for the candidate to complete the skills test by following on-screen instructions;

[1440] a means for analyzing the skills test results and biometric data in real time to generate an overall score for the candidate; and

[1441] A way for companies to create a list of the best candidates based on pre-defined criteria,

[1442] A means to provide companies with the best candidates,

[1443] a means for collecting biometric data of advertisement viewers to personalize advertisements;

[1444] A system that includes a means to measure the effectiveness of advertising in real time and provide feedback on the results.

[1445] (Claim 2)

[1446] a means using cameras, microphones, and sensors to collect biometric data of the candidate;

[1447] 10. The system of claim 1, including means for using cameras, microphones, and sensors to collect ad viewer responses.

[1448] (Claim 3)

[1449] a means of quantifying the candidate's skill level based on the results of the skills test; and

[1450] 10. The system of claim 1, further comprising means for quantifying the effectiveness of the advertisement based on response data of the advertisement viewers.

[1451] "Example 2: Combining Emotion Engines"

[1452] (Claim 1)

[1453] A means to store the skill sets, aptitude conditions, and evaluation criteria provided by the company in a database,

[1454] A means of registering various skill tests and tasks in a database and generating an individual assessment profile for each candidate;

[1455] A means for receiving and analyzing candidate skill test response data in real time;

[1456] means for analyzing biometric data and recognizing emotions of the candidate using an emotion engine;

[1457] a means for integrating the skill assessment, biometric assessment, and emotional assessment to generate a composite score;

[1458] A way to compare the overall score with the company's requirements to shortlist the best candidates,

[1459] A system that includes a means of providing companies with the best candidates.

[1460] (Claim 2)

[1461] 10. The system of claim 1, further comprising means for using a camera, a microphone, and a sensor to collect biometric data of the candidate.

[1462] (Claim 3)

[1463] 2. The system according to claim 1, further comprising means for quantifying the skill level of the candidate based on the results of the skill test and generating an evaluation profile.

[1464] "Application example 2 when combining emotion engines"

[1465] (Claim 1)

[1466] The means of collecting biometric data to assess candidates' skills, aptitudes and personality;

[1467] a means for the candidate to complete the skills test by following on-screen instructions;

[1468] a means for analyzing the skills test results and biometric data in real time to generate an overall score for the candidate; and

[1469] A means of matching registered evaluation criteria and listing the candidates who best fit the conditions,

[1470] A means to provide companies with the best candidates,

[1471] A means of assessing candidates' emotions and stress levels by analyzing facial expressions and voice in real time using cameras, microphones, and sensors;

[1472] A system that includes a means of presenting evaluation results using a comprehensive score based on customer service attitude and stress tolerance set by the company.

[1473] (Claim 2)

[1474] 10. The system of claim 1, comprising a camera, a microphone, a sensor, and an emotion recognition model for collecting biometric data of the candidate.

[1475] (Claim 3)

[1476] 10. The system of claim 1, further comprising means for quantifying the candidate's skill level and emotional stability based on the results of the skill test and real-time emotional analysis. [Explanation of symbols]

[1477] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. The means of collecting biometric data to assess candidates' skills, aptitudes and personality; a means for the candidate to complete the skills test by following on-screen instructions; a means for analyzing the skills test results and biometric data in real time to generate an overall score for the candidate; and A way for companies to create a list of the best candidates based on pre-defined criteria, A system that includes a means of providing companies with the best candidates.

2. 10. The system of claim 1, further comprising means for using a camera, a microphone, and a sensor to collect biometric data of the candidate.

3. 2. The system of claim 1, further comprising means for quantifying the skill level of the candidate based on the results of the skill test.

Images