Information processing systems, programs, and information processing methods

Abstract

This invention provides an information processing system, program, and method for improving the accuracy of recruitment selection using generative models. [Solution] The system includes an acquisition unit 101 that acquires a response from a job seeker to a first question for estimating the job seeker's skill level, and a dialogue unit 102 that, if the response does not satisfy predetermined conditions, inputs an instruction to generate a second question different from the first question into a generation model to acquire a second question. The acquisition unit 101 acquires the response from the job seeker to the second question, and the dialogue unit 102, if the response to the second question satisfies the conditions, inputs an evaluation instruction to the generation model along with the response to determine the skill level obtained by evaluating the skill level based on the response to determine the job seeker's confirmed skill level.

Description

Technical Field

[0001] The present invention relates to an information processing system, a program, and an information processing method.

Background Art

[0002] Conventionally, employment interviews using AI (artificial intelligence) have been conducted. In Patent Document 1, for application documents of applicants applying for employment by a business operator, by inputting a request to assign scores to each of a plurality of evaluation items to an AI chatbot, a technique for obtaining scores for each of the plurality of evaluation items output from the AI chatbot is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, improvement in accuracy in personnel evaluation using a generative model has been desired.

[0005] The present invention has been made in consideration of such points, and aims to improve the accuracy in personnel evaluation using a generative model.

Means for Solving the Problems

[0006] The information processing system of the present invention includes an acquisition unit that acquires a response sentence from the subject with respect to a first question sentence for estimating the skill level of the subject, a dialogue unit that, when the response sentence does not satisfy a predetermined condition, inputs a generation instruction for a second question sentence different from the first question sentence to a generative model to acquire the second question sentence, and includes The acquisition unit acquires the response text from the subject to the second question text, The dialogue unit is characterized in that, when the answer to the second question satisfies the conditions, it inputs an evaluation instruction to evaluate the skill level based on the answer to the generation model along with the answer, thereby determining the skill level obtained as the subject's confirmed skill level.

[0007] In the information processing system of the present invention, The aforementioned conditions may also relate to the credibility of the skill level obtained from the aforementioned response.

[0008] In the information processing system of the present invention, The aforementioned definition of creditworthiness may be set or changed according to user operations.

[0009] In the information processing system of the present invention, The aforementioned condition may also be that the creditworthiness obtained by inputting the creditworthiness determination criteria into the generative model along with the response text is equal to or greater than a threshold.

[0010] In the information processing system of the present invention, The dialogue unit may obtain the second question sentence by inputting the generation instruction and the fact that the second question sentence is a question intended to obtain a higher level of credibility, along with the answer sentence obtained by the acquisition unit, into the generation model.

[0011] In the information processing system of the present invention, The dialogue unit may generate the first question sentence by inputting the definitions for each skill level into a generative model.

[0012] In the information processing system of the present invention, The definitions for each skill level may be set or changed according to user operations.

[0013] In the information processing system of the present invention, the dialogue unit inputs a presentation instruction for presenting the reason for evaluating the skill level to the generation model, and thereby obtains, from the generation model, the reason for evaluating the skill level together with the skill level, and may present the determined skill level and the reason for evaluation.

[0014] In the information processing system of the present invention, the dialogue unit inputs a presentation instruction for presenting the basis of the reason for evaluation to the generation model, and thereby further obtains, from the generation model, a portion of the response sentence that serves as the basis of the reason for evaluation, and may present the skill level, the reason for evaluation, and the portion serving as the basis.

[0015] In the information processing system of the present invention, the dialogue unit may further cause the generation model to generate the first question sentence by inputting characteristics of the target person whose skill level is equal to or higher than a threshold value.

[0016] In the information processing system of the present invention, the target person is a job seeker, the skill level is the skill level with respect to the recruitment content of a predetermined employer, the dialogue unit may further cause the generation model to generate the first question by inputting characteristics of an applicant who has received an offer from the employer.

[0017] In the information processing system of the present invention, the target person is a job seeker, the skill level is the skill level with respect to the recruitment content of a predetermined employer, the dialogue unit may further cause the generation model to generate the first question by inputting information of the employer.

[0018] In the information processing system of the present invention, The subject is a job seeker, The dialogue unit may generate the first question by further inputting the information of the job seeker into the generation model.

[0019] In the information processing system of the present invention, The first question sentence may be a question sentence for evaluating the skill level for one item.

[0020] In the information processing system of the present invention, The first question sentence may be a question sentence for evaluating the skill level for each of a plurality of items.

[0021] The program of the present invention is A program that causes a computer to function as an acquisition unit and a dialogue unit, The acquisition unit acquires a response sentence from the subject for a first question sentence for estimating the skill level of the subject, When the response sentence does not satisfy a predetermined condition, the dialogue unit inputs a generation instruction for a second question sentence different from the first question sentence into the generation model to obtain a second question sentence, The acquisition unit acquires a response sentence from the subject for the second question sentence, When the response sentence for the second question sentence satisfies the condition, the dialogue unit determines the skill level obtained by inputting an evaluation instruction for evaluating the skill level based on the response sentence into the generation model together with the response sentence as the confirmed skill level of the subject.

[0022] An information processing method performed by a computer having a control unit, The step of the control unit acquiring a response sentence from the subject for a first question sentence for estimating the skill level of the subject, The step of the control unit obtaining a second question sentence by inputting a generation instruction for a second question sentence different from the first question sentence into the generation model when the response sentence does not satisfy a predetermined condition, The control unit performs the steps of obtaining the response from the subject to the second question, The control unit is characterized by including the step of determining the skill level obtained by inputting an evaluation instruction to the generation model, along with the answer to the second question, to evaluate the skill level based on the answer, when the answer satisfies the conditions, as the confirmed skill level of the subject. [Effects of the Invention]

[0023] According to the information processing system, program, and information processing method of the present invention, the accuracy of personnel evaluation using generative models is improved. [Brief explanation of the drawing]

[0024] [Figure 1] This is an overall diagram of the recruitment management system. [Figure 2] This figure shows an example of the data structure of an item list. [Figure 3] This figure shows an example of the data structure of creditworthiness information. [Figure 4] This is a flowchart of the selection process. [Figure 5] This is a flowchart of the selection process. [Figure 6] This is a flowchart of the selection process. [Figure 7] This is a flowchart of the selection process. [Figure 8] This is a flowchart illustrating the selection process according to the second embodiment. [Figure 9] This is a flowchart of the evaluation process. [Modes for carrying out the invention]

[0025] (First Embodiment) The first embodiment will be described below with reference to the drawings. Figure 1 is an overall configuration diagram of the recruitment management system 1 according to this embodiment. The recruitment management system 1 is an information processing system that manages the recruitment selection of job seekers using a generative model.

[0026] The recruitment management system 1 comprises a recruitment management device 10, a job seeker terminal 20, a recruiting company terminal 30, and a generation model device 40. The recruitment management device 10, the job seeker terminal 20, the recruiting company terminal 30, and the generation model device 40 are connected to each other via a communication network N such as the Internet.

[0027] The job seeker terminal 20 is a device used by job seekers. The job seeker terminal 20 transmits job seeker information, such as application documents like resumes and work history, to the recruitment management device 10. The recruiting company terminal 30 is a device used by a person in charge at a recruiting company. The recruiting company terminal 30 transmits recruiting company information, such as job requirements, and information about successful candidates, such as characteristics of successful candidates, to the recruitment management device 10. The recruitment management device 10 processes the recruitment of job seekers based on the job information and job seeker information. The recruitment management device 10 is an example of an information processing system. The generation model device 40 stores the generation model 410. The recruitment management device 10 processes the recruitment through interaction with the generation model 410.

[0028] The recruitment management device 10 is a computer or the like, and performs AI-based selection of job seekers by interacting with the generation model 410 stored in the generation model device 40. The recruitment management device 10 is composed of a computer or the like and includes a control unit 100, a storage unit 110, a communication unit 120, and a user interface (UI) unit 130.

[0029] The control unit 100 includes a processor such as a CPU (Central Processing Unit) and controls the operation of the recruitment management device 10. The storage unit 110 includes, for example, an HDD (Hard Disk Drive), RAM (Random Access Memory), ROM (Read Only Memory), and SSD (Solid State Drive). Furthermore, the storage unit 110 is not limited to being built into the recruitment management device 10, but may be a storage medium that can be detachably attached to the recruitment management device 10 (for example, a USB memory). In this embodiment, the storage unit 110 stores various programs executed by the control unit 100 and various data.

[0030] The communication unit 120 includes a communication interface for communicating with external devices wirelessly or via wire. The control unit 100 transmits and receives data between the job seeker terminal 20, the recruiting company terminal 30, and the generation model device 40 via the communication unit 120. The UI unit 130 includes an operation unit for receiving user operations, a display unit for displaying various information, a microphone for picking up speech and the like, and a speaker for emitting sound.

[0031] The job seeker terminal 20 comprises a control unit 200, a storage unit 210, a communication unit 220, and a UI unit 230. The control unit 200, storage unit 210, communication unit 220, and UI unit 230 are the same as the control unit 100, storage unit 110, communication unit 120, and UI unit 130 of the recruitment management device 10, respectively.

[0032] The recruiting company terminal 30 comprises a control unit 300, a storage unit 310, a communication unit 320, and a UI unit 330. The control unit 300, storage unit 310, communication unit 320, and UI unit 330 are the same as the control unit 100, storage unit 110, communication unit 120, and UI unit 130 of the recruitment management device 10, respectively. The job seeker terminal 20 and the recruiting company terminal 30 may be portable devices such as smartphones or PC tablets.

[0033] The generative model device 40 is a computer or the like. The generative model device 40 stores a generative model 410 that takes a question as input and outputs a result. The generative model 410 is a learning model generated by machine learning and generates a result in response to a question (prompt) from a user who accesses it via the communication network N. The generative model can be a Generative Pre-trained Transformer or a language model based thereon. Examples of generative models 410 include ChatGPT, GPT-3, GPT-4 by OpenAI, and GPT-J by EleutherAI.

[0034] The generative model 410 learns from a vast amount of data and then makes predictions for new inputs. The generative model device 40 hosts the generative model 410, calls it upon request, and returns the results. When the generative model 410 receives a prompt, which is an instruction statement from the recruitment management device 10, it performs calculations to generate a result corresponding to the prompt. The generated result is then sent to the recruitment management device 10.

[0035] Next, the configuration of the control unit 100 of the recruitment management device 10 will be described. The control unit 100 functions as an acquisition unit 101 and an interaction unit 102 by executing a program stored in the storage unit 110. In the following, the processes described as being performed by the acquisition unit 101 and the interaction unit 102 are processes performed by the control unit 100 by executing a program.

[0036] In the AI ​​selection process, the acquisition unit 101 acquires the job seeker's responses to the questions generated by the generation model 410 from the job seeker terminal 20 via the communication unit 120. The responses transmitted from the job seeker terminal 20 are assumed to be text data. Alternatively, the responses transmitted from the job seeker terminal 20 may be voice data. If the responses transmitted from the job seeker terminal 20 are voice data, the control unit 100 converts the voice data into text data through speech recognition processing, and the acquisition unit 101 acquires the text data.

[0037] The dialogue unit 102 transmits a prompt, which is an instruction to the generation model 410, via the communication unit 120. The dialogue unit 102 then obtains a response to the prompt. For example, in response to a prompt instructing the generation model 410 to create interview questions for a job applicant, the generation model 410 creates the interview questions, and the dialogue unit 102 obtains these questions as a response.

[0038] Next, we will explain the recruitment process. For example, when a recruiting company presents job requirements and job seekers submit application documents, a document screening is conducted. As the next step, instead of an interview by a recruiter, or as a separate selection process from the interview, an AI selection is carried out. In the AI ​​selection, confirmation of several pre-set items is performed by interacting with the generative model 410. Note that the recruitment process may include multiple stages.

[0039] The following describes the AI ​​selection process using the generative model 410. In the AI ​​selection process, the item list and creditworthiness information are referenced. This information is stored in the memory unit 110.

[0040] Figure 2 shows an example of an item list. The item list shows a list of each item to be checked in the AI ​​selection process, the definition of each item, and information showing the corresponding skill level and its definition. As shown in Figure 2, in this embodiment, the items are classified into three levels from the highest level: major items, medium items, and minor items. Furthermore, multiple medium items are set for each major item. In the example in Figure 2, the major item "(1) Results-Oriented" is shown to have three medium items: "(1-1) Achievement-Focused," "(1-2) Elimination of Uncertainty," and "(1-3) Proactiveness."

[0041] Furthermore, multiple sub-items are set for each major item. In the example in Figure 2, three sub-items are shown for "(1-1) Emphasis on Achievement": "(A) Motivation and Action Towards Achievement," "(B) Scope of Impact of Actions Towards Achievement," and "(C) Degree of Innovation." In addition, definitions are set for each major item, major item, and sub-item. The number of levels in these items, the number of items in each level, and the content of each item are assumed to be predetermined by the person in charge at the recruiting company or the administrator of the recruitment management device 10. Moreover, the number of levels in these items, the number of items in each level, and the content of each item are arbitrary and can be freely set.

[0042] In AI-based selection, a skill level is identified for each of the most subdivided items (sub-items in this embodiment). Here, the skill level is an evaluation value indicating the degree to which a candidate possesses the ability for that item. For example, for the sub-item "(A) Motivation and Action Towards Achievement" belonging to the medium-term item "(1-1) Emphasis on Achievement," five skill levels from 1 to 5 are set. A higher number indicates a higher skill level. Furthermore, a definition is pre-set for each skill level. The definition of the skill level is set to reflect the job seeker's experience. By judging skill levels in accordance with experience in this way, a highly reliable skill level can be obtained. The number of skill level levels and the definition of each skill level are pre-set by the administrator of the recruitment management device 10 or the person in charge at the recruiting company. Moreover, the number of skill level levels and the definition of each level are arbitrary and can be freely set.

[0043] In AI-based selection, the skill level of each applicant is identified through dialogue, and the reliability of this skill level is then evaluated. A reliability index is used for this evaluation. A higher reliability value indicates a higher probability that the skill level is reliable.

[0044] Figure 3 shows an example of creditworthiness information. Creditworthiness information includes creditworthiness values ​​and definitions (judgment criteria) for each value. In the example in Figure 3, creditworthiness is set to five levels, from "Creditworthiness 1" to "Creditworthiness 5". The more specific the content, such as mentioning the duration of work on a project and numerical results, the higher the creditworthiness. A definition is pre-set for each creditworthiness value. By setting definitions for creditworthiness in this way, it is possible to clarify what judgment criteria correspond to the creditworthiness value. The number of creditworthiness values ​​(number of levels) and their definitions are to be pre-set by the administrator of the recruitment management device 10 or the person in charge at the recruiting company. Furthermore, the number of creditworthiness levels and their definitions are arbitrary and can be freely set.

[0045] Figures 4 to 7 are flowcharts of the selection process related to AI selection. First, the dialogue unit 102 provides preconditions to the generation model 410 (step S100). Here, the preconditions include an item list, creditworthiness information, information on the target recruiting company, characteristics of successful candidates at the recruiting company, and information on job seekers. The recruiting company information is information about the recruiting company, and more specifically, information that can be referenced in matching applicants' requirements with the recruiting company's requirements during recruitment. The recruiting company information includes the recruiting company's philosophy, culture, business content, company information, and recruitment requirements. The characteristics of successful candidates may be information indicating the characteristics of applicants who have already been offered positions, or it may be information indicating the characteristics of successful candidates that the recruiting company is looking for, i.e., the characteristics of ideal successful candidates. The job seeker information is information from the job seeker's resume and work history. The dialogue unit 102 instructs the generation model 410, along with the preconditions, to conduct the recruitment selection based on these.

[0046] Next, the dialogue unit 102 selects a sub-item to be processed (step S102). Specifically, the dialogue unit 102 selects one of the major items as the item to be processed, and then selects the first sub-item included in the major item as the item to be processed. Then, the dialogue unit 102 selects the first sub-item included in the sub-item as the item to be processed. In the following explanation, we will use the example where the major item "(1) Results-Oriented," the sub-item "(1-1) Achievement-Oriented," and the sub-item "(A) Motivation and Action Towards Achievement" are selected as the items to be processed.

[0047] Next, the dialogue unit 102 obtains the initial question text corresponding to the sub-item selected as the processing target from the generation model 410 (step S104). Here, the initial question text is a question text for estimating the job seeker's skill level and trustworthiness for the sub-item being processed. Specifically, the dialogue unit 102 requests the generation model 410 to generate the initial question text by sending a prompt to the generation model device 40 instructing it to generate the initial question text corresponding to one sub-item selected as the processing target. When the generation model device 40 receives the prompt, it inputs it to the generation model 410. As a result, the generation model 410 generates the initial question text. The generation model device 40 sends the initial question text to the recruitment management device 10. The dialogue unit 102 then obtains the initial question text as a response to the transmitted prompt. For example, the following text can be obtained as the initial question text. "Please describe an instance from your previous work experience where you proactively planned and adjusted your actions to achieve a given goal. Please elaborate on the challenges you faced, the specific measures you took, and the results."

[0048] Next, the dialogue unit 102 presents the initial question to the job seeker (step S106). Specifically, the dialogue unit 102 transmits the initial question to the job seeker terminal 20 used by the job seeker via the communication unit 120. On the job seeker terminal 20, the control unit 200, upon receiving the initial question, displays it on the display unit of the UI unit 230. This allows the job seeker to confirm the initial question generated by the generation model 410. After confirming the initial question, the job seeker inputs a response using the operation unit of the job seeker terminal 20. This response is transmitted from the job seeker terminal 20 to the recruitment management device 10. For example, the following text may be obtained as a response. "For me, there's a clear work standard: delivering work at least one or two days before the deadline is a must. So, I break down tasks into smaller parts and create a schedule with the goal of delivering one or two days in advance. ...This allows me to notice any missed tasks before delivery and increases my credibility with clients by submitting work early."

[0049] In the recruitment management device 10, the control unit 100 obtains the response text from the job seeker via the communication unit 120 (step S108). The control unit 100 displays the response text on the display unit of the UI unit 130. Alternatively, the control unit 100 may transmit the response text to the recruiting company terminal 30 and have it displayed on the display unit of the recruiting company terminal 30.

[0050] Next, the dialogue unit 102 obtains the skill level and credibility score corresponding to the response sentence from the generation model 410 (step S110). Specifically, the dialogue unit 102 requests the generation model device 40 to perform an evaluation of the skill level and credibility score based on the response sentence, along with the response sentence. When the generation model device 40 receives the prompt, it inputs it to the generation model 410. As a result, the generation model 410 calculates the skill level and credibility score according to the prompt and outputs them. The skill level and credibility score are sent to the recruitment management device 10. The recruitment management device 10 then obtains the skill level and credibility score as evaluation results from the dialogue unit 102. For example, for the response sentence example above, a skill level of 3 and a credibility score of 4 are obtained.

[0051] Next, the dialogue unit 102 presents the obtained skill level and credibility score (S112). Specifically, the dialogue unit 102 displays the skill level and credibility score on the display unit of the UI unit 130 of the recruitment management device 10. This allows the user of the recruitment management device 10 to confirm the skill level and credibility score.

[0052] As another example, the skill level and credibility score may be presented to the user of the recruiting company terminal 30. In this case, the dialogue unit 102 transmits the skill level and credibility score to the recruiting company terminal 30 via the communication unit 120. Then, on the recruiting company terminal 30, the control unit 300 acquires the skill level and credibility score and displays it on the display unit of the UI unit 330. This allows the recruiting company to also confirm the skill level and credibility score.

[0053] Next, the dialogue unit 102 checks whether the response is negative (step S114). A response is negative if it contains a sentence that negates the content of the question, such as "We met the deadline, but we did not make a plan for it." In the case of such a negative sentence, it is highly likely that the requested content is not included, and therefore it is determined that an evaluation cannot be made from this response. The generative model 410 will determine whether the response is negative or not. Specifically, the prompt input to the generative model 410 includes outputting information to that effect if the response is negative. If the dialogue unit 102 determines that the response is negative (Y in step S114), it proceeds to step S140.

[0054] If the response is not in the negative form (N in step S114), the dialogue unit 102 checks the credibility of the skill level obtained for the response (step S116). If the credibility is above a predetermined threshold (Y in step S116), the dialogue unit 102 proceeds to step S120 shown in Figure 5. The threshold is assumed to be predetermined. For example, a credibility of 3 is set as the threshold.

[0055] If the credibility is above the threshold, the skill level obtained for the response is treated as a reliable value and is used in the processing from step S120 onward. In step S120, the dialogue unit 102 checks whether a provisional skill level has been set. Here, the provisional skill level is a skill level that is set provisionally before the skill level is finalized. In the initial state, the provisional skill level has no value set.

[0056] If no provisional skill level is set (N in step S120), the dialogue unit 102 proceeds to step S124. If a provisional skill level is set (Y in step S120), the dialogue unit 102 compares the skill level (acquired skill level) obtained from the generation model 410 in the most recent step S110 with the provisional skill level (step S122). If the acquired skill level is higher than the provisional skill level (Y in step S122), the dialogue unit 102 sets the acquired skill level to the provisional skill level (step S124). If the provisional skill level is in its initial state and the acquired skill level is skill level 3, the provisional skill level is set to 3.

[0057] The dialogue unit 102 does not update the provisional skill level if the acquired skill level is the same as or lower than the provisional skill level (N in step S122). Specifically, in this case, the dialogue unit 102 proceeds to step S140. In this selection process, the provisional skill level may be updated, but once a provisional skill level has been set, it will not be updated to a lower level. This is to prevent the amount of computation from becoming too large by preventing the loop processing from being repeated.

[0058] After processing in step S124, the dialogue unit 102 checks whether there is a skill level higher than the provisional skill level set for the sub-item being processed (step S126). For example, for the sub-item "(A) Motivation and initiative to achieve," skill levels from 1 to 5 are defined. Therefore, if a value of 4 or less is set for the provisional skill level, it is determined that there is a skill level higher than the provisional skill level.

[0059] If the dialogue unit 102 finds that there is a skill level higher than the provisional skill level (Y in step S126), it obtains a level-up question from the generation model 410 (step S128). Here, a level-up question is a question used to confirm whether the job seeker possesses a higher skill level. Specifically, the dialogue unit 102 requests the generation model 410 to generate a level-up question by sending a prompt instructing the generation of a level-up question for the job seeker. When the generation model device 40 receives the prompt, it inputs it to the generation model 410. The generation model 410 generates a level-up question based on the prompt and the information obtained so far.

[0060] The generation model device 40 transmits the level-up question text generated by the generation model 410 to the adoption management device 10. The dialogue unit 102 then retrieves the level-up question text as a response to the transmitted prompt.

[0061] For example, if the provisional skill level is set to "3", the level-up question might look something like this: "Please tell us about any business efficiency ideas you have proposed or implemented that have contributed to reducing organizational working hours or costs. Please also describe the results achieved."

[0062] Next, the dialogue unit 102 presents the job seeker with a level-up question (step S130). Then, the dialogue unit 102 resets the in-depth question count to zero (step S132). The in-depth question count will be explained later.

[0063] The control unit 100 then proceeds to step S108. From step S108 onward, the control unit 100 processes according to the response entered by the job seeker. Suppose that the response obtained in step S108 is not negative, has a confidence level above the threshold, and yields a skill level higher than the provisional skill level. In this case, in step S124, the provisional skill level is updated to a higher level. In this way, the processing from step S108 to step S132 is repeated, and the provisional skill level is updated to a higher level.

[0064] Furthermore, if the skill level is below the provisional skill level in step S122 (N in step S122), the dialogue unit 102 proceeds to step S140 shown in Figure 6. In step S140, the dialogue unit 102 sets the value set as the provisional skill level at the time of processing as the final skill level. Also, if the provisional skill level is not set at the time of processing, the dialogue unit 102 sets information indicating that evaluation is not possible for the final skill level.

[0065] Furthermore, in step S126 shown in Figure 5, if there is no skill level higher than the provisional skill level (N in step S126), the dialogue unit 102 proceeds to step S140. In this case as well, the dialogue unit 102 sets the value set as the provisional skill level at the time of processing as the final skill level in step S140. That is, in this case, the dialogue unit 102 sets the highest value of the skill level for the sub-item being processed as the final skill level. Since the highest value of the skill level for "(A) Motivation and initiative to achieve" is "5", "5" is set as the final skill level.

[0066] Furthermore, as mentioned above, in step S114 shown in Figure 4, if the response is negative (Y in step S114), the dialogue unit 102 proceeds to step S140. In this case as well, in step S140, the dialogue unit 102 either sets the value set as the provisional skill level at the time of processing as the confirmed skill level, or sets information indicating that evaluation is not possible. Thus, if the response is negative, the dialogue unit 102 terminates the skill level check at this point.

[0067] After the processing in step S140, the dialogue unit 102 presents the confirmed skill level (S142). Specifically, the dialogue unit 102 displays the confirmed skill level on the display unit of the UI unit 130. Alternatively, the dialogue unit 102 may display the confirmed skill level on the display unit of the UI unit 330 of the recruiting company terminal 30.

[0068] The dialogue unit 102 checks if the following sub-items exist (step S144). For example, if the sub-item to be processed is "(A) Motivation and initiative to achieve," then it is determined that the following sub-items exist because no other sub-items have been selected in the main item "(1-1) Emphasis on achievement."

[0069] If the dialogue unit 102 has the following sub-item (Y in step S144), it selects the next sub-item to be processed (step S146). Next, the dialogue unit 102 resets the value of the in-depth question count to zero (step S148), and then proceeds to step S104. In this case, the dialogue unit 104 continues processing the newly selected sub-item from step S104 onward.

[0070] In step S142, if there are no further sub-items (N in step S144), the dialogue unit 102 completes the process, considering that processing has been completed for all sub-items belonging to the target medium item. This process is performed for all medium items belonging to all major items included in the item list.

[0071] On the other hand, if the credibility of the skill level obtained for the answer in step S110 is below the threshold (N in step S116), the skill level cannot be determined as is. In this case, the dialogue unit 102 presents the job seeker with further questions in the processing from step S150 onwards, as shown in Figure 7. Here, the further questions are questions designed to further explore the content of the job seeker's answer, and these questions will hereafter be referred to as in-depth questions. In-depth questions are questions designed to obtain a skill level with a higher credibility than the previous answer. Preferably, in-depth questions are questions that elicit answers based on the job seeker's actual experience.

[0072] Specifically, in step S116, if the credibility is below a threshold or if the credibility cannot be evaluated (N in step S116), the dialogue unit 102 proceeds to step S150 shown in Figure 7. In step S150, the dialogue unit 102 checks the in-depth question count. The in-depth question count is initially set to zero and is incremented each time an in-depth question is generated. The in-depth question count also has a preset maximum count value.

[0073] If the in-depth question count is less than the maximum count (Y in step S150), the dialogue unit 102 obtains an in-depth question from the generation model 410 (step S152). Specifically, the dialogue unit 102 requests the generation model 410 to generate an in-depth question by sending a prompt instructing the generation of an in-depth question for the job seeker. The prompt includes information that the in-depth question is designed to obtain a higher level of skill than the answers obtained so far. More preferably, the prompt includes information that the in-depth question is designed to elicit answers based on the job seeker's actual experience.

[0074] In the generation model device 40, upon receiving a prompt, it inputs it to the generation model 410. Based on the prompt and the information obtained so far, the generation model 410 generates a follow-up question. The follow-up question is then sent to the recruitment management device 10.

[0075] In response to this, the dialogue unit 102 obtains a follow-up question as an answer to the prompt. Next, the dialogue unit 102 presents the follow-up question to the job seeker (step S154).

[0076] For example, suppose the following response was obtained in response to the initial question. "I made a schedule and broke down the tasks into smaller steps." In this case, the credibility is rated as 2, and it is determined that further questioning is necessary. In this case, for example, the following further question sentences may be obtained. "When you created a schedule and broke down the tasks, what specific actions did you take based on that plan? What results did you achieve? Please also tell us about your goals and criteria when you undertook the task."

[0077] Then, the dialogue unit 102 increments the in-depth question count by 1 (step S156). The control unit 100 then proceeds to step S108. From step S108 onward, the control unit 100 proceeds with processing according to the answer entered by the job seeker.

[0078] Furthermore, in step S150, if the count of in-depth questions is greater than or equal to the maximum count (N in step S150), the process proceeds to step S140 to determine the skill level.

[0079] In this way, if the credibility level does not meet the threshold, the dialogue unit 102 automatically generates follow-up questions and presents them to the job seeker, thereby obtaining further responses from the job seeker. Therefore, the skill level and credibility level can be re-evaluated based on the newly obtained responses.

[0080] As described above, in the recruitment management system 1 of this embodiment, if the reliability of the answer obtained for a single question is low, the recruitment management device 10 generates a second question that delves deeper into the topic, and allows the applicant to input an answer again. This makes it possible to obtain a more reliable answer. Furthermore, the recruitment management device 10 can obtain information from the job seeker that is necessary to determine their skill level by presenting a new question as a second question that corresponds to the skill level, depending on the content of the answer. Thus, the recruitment management system 1 of this embodiment can improve the accuracy of talent evaluation in recruitment selection using a generative model.

[0081] In this embodiment, since definitions are set for each skill level, users can understand the meaning of a skill level by referring to its definition when it is presented. Therefore, more detailed information can be obtained than if only numerical values ​​were presented. Furthermore, the definitions for each skill level are set so that the skill level is judged in accordance with the job seeker's experience. Therefore, a more reliable skill level can be obtained.

[0082] (Second Embodiment) Next, the differences between the recruitment management system 1 according to the second embodiment and the first embodiment will be mainly described. In the second embodiment, the dialogue unit 102 of the recruitment management device 10 causes the generation model 410 to generate a question sentence as the initial question sentence, which allows evaluation of the skill level and corresponding creditworthiness for each of the multiple sub-items belonging to one main item.

[0083] Figure 8 is a flowchart of the selection process according to the second embodiment. First, the dialogue unit 102 provides preconditions to the generation model 410 (step S200). This process is the same as the process in step S100, which was described with reference to Figure 4.

[0084] Next, the dialogue unit 012 selects the sub-items to be processed (step S202). Specifically, the dialogue unit 102 selects one of the major items as the item to be processed, and then selects the first sub-item included in the major item as the item to be processed. In the following explanation, we will use the example where the major item "(1) Results-Oriented" and the sub-item "(1-1) Achievement-Focused" are selected as the items to be processed.

[0085] Next, the dialogue unit 102 obtains the initial question for the medium item selected as the processing target from the generation model (step S204). Specifically, the dialogue unit 102 requests the generation model 410 to generate the initial question by sending a prompt instructing the generation of the initial question corresponding to the selected medium item. This prompt shall include information specifying that the initial question evaluates the skill level and credibility of all sub-items included in the medium item to be processed.

[0086] In the generation model device 40, upon receiving a prompt, it inputs it to the generation model 410. This causes the generation model to generate the initial question. The generation model device 40 then sends the initial question to the adoption management device 10. The dialogue unit 102 then retrieves the initial question as a response to the transmitted prompt. For example, the following sentence may be obtained as the initial question. "Please tell me about a specific challenge you faced in your previous workplace. Please explain in detail how you dealt with that challenge, what actions you took, and what results those actions produced."

[0087] Next, the dialogue unit 102 presents the initial question to the job seeker (step S206). As a result, the job seeker terminal 20 inputs a response, and the response is transmitted to the recruitment management device 10. In the recruitment management device 10, the control unit 100 obtains the response from the job seeker via the communication unit 120 (step S208). For the above question, an answer such as the following can be obtained. "We had a project where we had to deliver several websites in a very short period of time. That's when I came up with the idea of ​​creating a CMC (Content Management System) to enable mass production of websites..."

[0088] Next, the dialogue unit 102 obtains the skill level and its credibility score corresponding to the response sentence from the generation model 410 (step S210). In this case, the dialogue unit 102 obtains the skill level and its credibility score for all sub-items belonging to the main item being processed.

[0089] Specifically, the dialogue unit 102 requests the generation model 410 to perform evaluations by sending a response along with an evaluation instruction prompt to the generation model 410, which in turn requests the evaluation of the skill level and credibility of all sub-items belonging to the main item being processed, based on the response. When the generation model device 40 receives the prompt, it inputs it to the generation model 410. As a result, the generation model 410 calculates the skill level and credibility of all sub-items belonging to the main item being processed according to the prompt and outputs them. The skill level and credibility are sent to the recruitment management device 10, and the dialogue unit 102 obtains the skill level and credibility for each sub-item as evaluation results.

[0090] For the above response, the following evaluation results can be obtained, for example. (A) Motivation and initiative to achieve: Skill level 4, Trust level 4 (B) Scope of impact of actions towards achievement: Skill level 3, Trust level 4 ...

[0091] Thus, in the second embodiment, the dialogue unit 102 can obtain the skill level and creditworthiness for all sub-items belonging to the main item being processed from a single response from the job seeker.

[0092] Next, the dialogue unit 102 presents the obtained skill level and credibility score (S212). Next, the dialogue unit 102 performs an evaluation process for all sub-items to be processed (step S214). Figure 9 is a flowchart showing the detailed processing in the evaluation process (step S214). Note that the evaluation process (step S214) is performed for each sub-item to be processed. In the evaluation process (step S214), the dialogue unit 102 first checks whether the answer statement is in the negative form or not (step S230). If the answer statement is in the negative form (Y in step S230), the dialogue unit 102 proceeds to step S232. In step S232, the dialogue unit 102 sets the value set as the provisional skill level at the time of processing as the final skill level. Also, if the provisional skill level has not been set at the time of processing, the dialogue unit 102 sets information indicating that evaluation is not possible in the final skill level. The dialogue unit 102 then terminates the evaluation process.

[0093] On the other hand, if the response is not in the negative form (N in step S230), the dialogue unit 102 checks the credibility of the skill level obtained for the response (step S234). If the credibility is below the threshold (N in step S234), the dialogue unit 102 determines that further questioning is necessary (step S236) and terminates the evaluation process.

[0094] If the credibility is above a preset threshold (Y in step S234), the dialogue unit 102 checks whether a provisional skill level has been set (step S238). If no provisional skill level has been set (N in step S238), the dialogue unit 102 proceeds to step S244. If a provisional skill level has been set (Y in step S238), the dialogue unit 102 compares the skill level obtained from the generation model 410 (acquired skill level) with the provisional skill level in step 210 (step S240).

[0095] If the acquired skill level is less than or equal to the provisional skill level (N in step S240), the dialogue unit 102 proceeds to step S242. In step S242, the dialogue unit 102 sets the value set as the provisional skill level at the time of processing as the final skill level. If the provisional skill level is not set at the time of processing, the dialogue unit 102 sets information indicating that evaluation is not possible for the final skill level. With this, the evaluation process is completed.

[0096] The dialogue unit 102 sets the acquired skill level to the provisional skill level (step S244) if the acquired skill level is higher than the provisional skill level (Y in step S240). Next, the dialogue unit 102 checks whether there is a skill level higher than the provisional skill level (step S246). If there is no skill level higher than the provisional skill level (N in step S246), the dialogue unit 102 terminates the evaluation process. If there is a skill level higher than the provisional skill level (Y in step S246), the dialogue unit 102 determines that a level-up question is necessary (step S248) and terminates the evaluation process.

[0097] In Figure 8, after the evaluation process (step S214) is performed, the dialogue unit 102 determines whether the evaluation has been completed for all sub-items belonging to the main item being processed (step S216). The evaluation is determined to be complete if a final skill level has been determined for all sub-items.

[0098] If the dialogue unit 102 has finished evaluating all sub-items (Y in step S216), it proceeds to step S218. If the dialogue unit 102 determines that follow-up questions are necessary (follow-up questions necessary in step S216), it retrieves follow-up question text from the generative model 410, presents it to the job seeker (step S220), and then proceeds to step S208. This process is the same as the processes in steps S152 and S154. In the processes from step S208 onward, the dialogue unit 102 retrieves the response text from the job seeker and, based on this, re-evaluates the skill level and trustworthiness.

[0099] Furthermore, if the dialogue unit 102 determines that a level-up question is necessary (a level-up question is necessary in step S216), it retrieves the level-up question text from the generative model 410, presents it to the job seeker (step S222), and then proceeds to step S208. This process is the same as the processes in steps S128 and S130. In the processes from step S208 onward, the dialogue unit 102 retrieves the response text from the job seeker and, based on this, re-evaluates the skill level and credibility. If the dialogue unit 102 determines that a level-up question is necessary and also determines that a follow-up question is necessary, the follow-up question will take precedence.

[0100] In step S218, the dialogue unit 102 checks whether there is a sub-item that has not been selected as the processing target. If the dialogue unit 102 finds that there is a sub-item that has not been selected as the processing target (Y in step S218), it proceeds to step S202. In this case, in step S202, the dialogue unit 102 selects the unselected sub-item as the processing target and proceeds with the subsequent processing. If there is no sub-item that has been selected as the processing target (N in step S218), the dialogue unit 102 terminates the selection process.

[0101] In the second embodiment of the recruitment management system 1, the accuracy of the selection process can be improved in the recruitment selection process using the generative model. Furthermore, in this embodiment, the recruitment management device 10 can obtain the skill level and creditworthiness corresponding to each of the multiple sub-items from a single response, thereby streamlining the selection process.

[0102] The embodiments described above are merely examples for carrying out the present invention, and various other embodiments can be adopted. For example, various modifications and changes are possible within the scope of the gist of the present invention as described in the claims, such as applying one modification to another. For example, some of the components of the above embodiments may be omitted, or the order of processing may be changed or omitted.

[0103] One such first modification is that if the evaluation result of the response entered by the job seeker does not meet predetermined conditions, the dialogue unit 102 may input a command to the generation model 410 to generate a different question from the previous one, and the specific processing for this is not limited to the embodiment. For example, if the dialogue unit 102 fails to obtain the information that was pre-set as the desired acquisition as a result of the evaluation of the response, it may input a command to the generation model 410 to generate a new question in order to obtain that information.

[0104] A second modification will now be described. In this embodiment, the information such as the question text presented by the dialogue unit 102 is displayed on the job seeker's display unit, but the output format of this information is not limited to this embodiment. As another example, it may be output as sound by a speaker.

[0105] As a third variation, in step S112, the dialogue unit 102 may present, along with the skill level and credibility, the evaluation reason and its basis for obtaining such an evaluation result. Here, the basis is the part of the response that was used as the basis for the judgment. In this case, the dialogue unit 102 inputs the evaluation reason and a prompt instructing the generation model 410 to present its basis. As a result, the dialogue unit 102 can obtain not only the skill level and credibility, but also the evaluation reason and its basis.

[0106] Examples of evaluation reasons and justifications include the following sentences. Note that the following sentences represent evaluation reasons corresponding to skill level 1 and credibility level 2. Reason for evaluation: "While the reasons and background for the specific actions are not sufficiently explained, the actions themselves are described." The reasoning is, "I made a schedule and broke down the tasks into smaller steps."

[0107] Similarly, in step S142, the dialogue unit 102 may present not only the confirmed skill level, but also the corresponding credibility, evaluation reason, and basis. In this case, in step S124, not only the acquired skill level, but also the credibility, evaluation reason, and basis corresponding to this acquired skill level will be set together with the confirmed skill level.

[0108] Furthermore, in step S212, the dialogue unit 102 may also present the skill level and credibility, along with the reasons for the evaluation and their basis.

[0109] As a fourth variation, although this embodiment describes the process for determining the skill level of job seekers in AI selection, the target is not limited to job seekers. The target can be any person whose skill level needs to be evaluated. For example, in cases where an employee is evaluated within a company, an information processing system may be used to evaluate the skill level of each employee. Furthermore, in this case, instead of job offer information, the characteristics of individuals (target persons) whose skill level is above a threshold are included as preconditions.

[0110] According to the information processing system, program, and information processing method of this embodiment, which have the above configuration, an acquisition unit 101 acquires a response from a subject to a first question for estimating the subject's skill level, and a dialogue unit 102 acquires a second question by inputting a generation instruction to a generation model for a second question different from the first question if the response does not satisfy predetermined conditions. The acquisition unit 101 acquires the response from the subject to the second question, and the dialogue unit 102, if the response to the second question satisfies the conditions, inputs an evaluation instruction to a generation model 410 along with the response to determine the skill level obtained by evaluating the skill level based on the response to be the subject's confirmed skill level. In this way, a second question is automatically generated and a corresponding response is obtained, which improves the accuracy of recruitment selection using the generation model.

[0111] Furthermore, in the information processing system, program, and information processing method of this embodiment, the conditions may also be conditions relating to the credibility of the skill level obtained from the answer text. This makes it possible to determine whether or not to generate a second question text according to the credibility of the skill level.

[0112] Furthermore, in the information processing system, program, and information processing method of this embodiment, the definition of trustworthiness may be set or changed according to user operation. This allows the definition of trustworthiness to be arbitrarily set or changed.

[0113] Furthermore, in the information processing system, program, and information processing method of this embodiment, the condition may be that the credibility obtained by inputting the credibility judgment criteria into the generation model along with the answer text is equal to or greater than a threshold. This makes it possible to automatically determine the credibility level of the skill level.

[0114] Furthermore, in the information processing system, program, and information processing method of this embodiment, the dialogue unit 102 may acquire the second question by inputting a generation instruction and a statement that the second question is intended to obtain a higher level of credibility, along with the answer text acquired by the acquisition unit 101, into the generation model 410. This allows the second question to be acquired automatically.

[0115] Furthermore, in the information processing system, program, and information processing method of this embodiment, the dialogue unit 102 may generate a first question statement by inputting definitions for each skill level into the generation model. This makes it possible to automatically obtain the first question statement.

[0116] Furthermore, in the information processing system, program, and information processing method of this embodiment, the definitions for each skill level may be set or changed according to user operation. This allows the definitions for each skill level to be set arbitrarily.

[0117] Furthermore, in the information processing system, program, and information processing method of this embodiment, the dialogue unit 102 may input a presentation instruction to the generation model 410 to present the reasons for evaluating the skill level, thereby obtaining the reasons for evaluating the skill level along with the skill level from the generation model 410, and presenting the determined skill level and the reasons for evaluation. This allows the user to confirm not only the skill level but also the reasons for evaluation.

[0118] Furthermore, in the information processing system, program, and information processing method of this embodiment, the dialogue unit 102 may input a presentation instruction to the generation model 410 to present the basis for the evaluation reason, thereby further obtaining the portion of the response that serves as the basis for the evaluation reason from the generation model 410, and presenting the skill level, evaluation reason, and the portion that serves as the basis for the description. This allows the user to confirm not only the skill level, but also the evaluation reason and the portion that serves as the basis for the description.

[0119] Furthermore, in the information processing system, program, and information processing method of this embodiment, the dialogue unit 102 may also generate a first question by inputting the characteristics of a subject whose skill level is above a threshold into the generation model. This makes it possible to obtain a first question that reflects the characteristics of the subject.

[0120] Furthermore, in the information processing system, program, and information processing method of this embodiment, the target person is a job seeker, the skill level is the skill level required for the recruitment content of a predetermined recruiting company, and the dialogue unit 102 may further input the characteristics of the successful candidate to the recruiting company into the generation model to generate the first question. This makes it possible to obtain a first question that reflects the characteristics of the successful candidate.

[0121] Furthermore, in the information processing system, program, and information processing method of this embodiment, the target person is a job seeker, the skill level is the skill level required for the recruitment content of a predetermined recruiting company, and the dialogue unit 102 may generate a first question by further inputting information of the recruiting company into the generation model 410. This makes it possible to obtain a first question sentence that reflects the information of the recruiting company.

[0122] Furthermore, in the information processing system, program, and information processing method of this embodiment, the target person is a job seeker, and the dialogue unit 102 may generate a first question by further inputting the job seeker's information into the generation model. This makes it possible to obtain a first question sentence that reflects the job seeker's information.

[0123] Furthermore, in the information processing system, program, and information processing method of this embodiment, the first question statement may be a question statement for evaluating the skill level for a single item. This makes it possible to obtain a first question statement for evaluating the skill level for each item.

[0124] Furthermore, in the information processing system, program, and information processing method of this embodiment, the first question statement may be a question statement for evaluating the skill level for each of the multiple items. This makes it possible to obtain a first question statement for evaluating the skill level for multiple items. [Explanation of symbols]

[0125] 1. Recruitment Management System 10 Recruitment management device 20 Job seeker terminals 30 Recruitment Provider Terminals 40 Generative Model Device 100 Control Unit 101 Acquisition Department 102 Dialogue Section 110 Storage section 120 Communications Department 130 UI section 200 Control Unit 210 Storage section 220 Communications Department 230 UI section 300 Control Unit 310 Storage section 320 Communications Department 330 UI section 410 Generative Models

Claims

1. An acquisition unit that acquires the response text from the subject to a first question for estimating the subject's skill level, If the aforementioned answer does not satisfy the predetermined conditions, the dialogue unit obtains a second question by inputting an instruction to generate a second question different from the first question into the generation model. Equipped with, The acquisition unit acquires the response text from the subject to the second question text, The dialogue unit is an information processing system that, when the answer to the second question satisfies the conditions, inputs an evaluation instruction to the generation model, along with the answer, to evaluate the skill level based on the answer, thereby determining the skill level obtained as the subject's confirmed skill level.

2. The information processing system according to claim 1, wherein the aforementioned conditions are conditions relating to the credibility of the skill level obtained from the aforementioned response.

3. The information processing system according to claim 2, wherein the definition of the creditworthiness can be set or changed according to user operation.

4. The information processing system according to claim 2, wherein the condition is that the creditworthiness obtained by inputting the creditworthiness determination criteria into the generation model along with the answer statement is equal to or greater than a threshold.

5. The information processing system according to claim 4, wherein the dialogue unit obtains the second question sentence by inputting the generation instruction and the fact that the second question sentence is a question for obtaining a higher level of credibility, along with the answer sentence obtained by the acquisition unit, into the generation model.

6. The information processing system according to claim 1, wherein the dialogue unit generates the first question sentence by inputting the definitions for each skill level into a generation model.

7. The information processing system according to claim 6, wherein the definitions for each skill level can be set or changed according to user operation.

8. The aforementioned dialogue unit, By inputting a presentation instruction that presents the reasons for evaluating the skill level into the generative model, the reasons for evaluating the skill level are obtained from the generative model along with the skill level. The determined skill level and the reasons for the evaluation are presented. The information processing system according to claim 6.

9. The aforementioned dialogue unit, By inputting instructions to the generative model that provide the basis for the evaluation reasons, the generative model further retrieves the portion of the response that serves as the basis for the evaluation reasons. The information processing system according to claim 8, which presents the skill level, the reason for the evaluation, and the basis for the evaluation.

10. The information processing system according to claim 6, wherein the dialogue unit further generates the first question sentence by inputting the characteristics of the subject whose skill level is above a threshold into the generative model.

11. The aforementioned subject is a job seeker, The aforementioned skill level is the skill level required for the recruitment content of a specified recruiting company. The information processing system according to claim 6, wherein the dialogue unit further inputs the characteristics of the person who has been offered a position by the recruiting company into the generation model to generate the first question sentence.

12. The aforementioned subject is a job seeker, The aforementioned skill level is the skill level required for the recruitment content of a specified recruiting company. The information processing system according to claim 6, wherein the dialogue unit further inputs the information of the recruiting business operator into the generation model to generate the first question sentence.

13. The information processing system according to claim 6, wherein the dialogue unit further inputs the subject's information into the generation model to generate the first question sentence.

14. The information processing system according to claim 1, wherein the first question is a question for evaluating the skill level for a single item.

15. The information processing system according to claim 1, wherein the first question is a question for evaluating the skill level for each of the multiple items.

16. A program that causes a computer to function as an acquisition unit and an interaction unit, The acquisition unit acquires the response text from the subject to a first question for estimating the subject's skill level. If the response sentence does not satisfy the predetermined conditions, the dialogue unit inputs an instruction to generate a second question sentence different from the first question sentence into the generation model, thereby obtaining a second question sentence. The acquisition unit acquires the response text from the subject to the second question text, The dialogue unit is a program that, when the answer to the second question satisfies the conditions, inputs an evaluation instruction to the generation model, along with the answer, to evaluate the skill level based on the answer, thereby determining the skill level obtained as the subject's confirmed skill level.

17. An information processing method performed by a computer having a control unit, The control unit performs the steps of obtaining the subject's response to a first question for estimating the subject's skill level, The control unit, if the answer statement does not satisfy predetermined conditions, inputs an instruction to generate a second question statement different from the first question statement into the generation model, thereby obtaining a second question statement. The control unit performs the steps of obtaining the response from the subject to the second question, Information processing method comprising the step of the control unit determining the skill level obtained by inputting an evaluation instruction to the generation model, along with the answer to the second question, to evaluate the skill level based on the answer, when the answer satisfies the conditions, as the confirmed skill level of the subject.

Images