Information processing system, program, and information processing method

Abstract

Provided is an information processing system, program, and information processing method that improve the accuracy of recruitment selection using a generative model. [Solution] The system includes an acquisition unit 101 that acquires an answer from the job seeker to a first question for estimating the job seeker's skill level, and a dialogue unit 102 that acquires the second question by inputting an instruction to generate a second question different from the first question into a generation model if the answer does not satisfy a predetermined condition. The acquisition unit 101 acquires the answer from the job seeker to the second question, and if the answer to the second question satisfies the condition, the dialogue unit 102 determines the skill level obtained by inputting, together with the answer, an evaluation instruction to evaluate the skill level based on the answer into the generation model as the final skill level of the job seeker.

Description

[Technical field]

[0001] The present invention relates to an information processing system, a program, and an information processing method. [Background technology]

[0002] Hitherto, AI (artificial intelligence) has been used in job interviews. Patent Document 1 discloses a technology for inputting a request to an AI chatbot to assign a score to each of a number of evaluation items in the application documents of an applicant for employment at a business, and acquiring scores for each of a number of evaluation items output from the AI ​​chatbot. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7525696 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a need to improve the accuracy of talent evaluation using generative models.

[0005] The present invention has been made in consideration of these points, and aims to improve the accuracy of talent evaluation using generative models. [Means for solving the problem]

[0006] The information processing system of the present invention comprises: an acquisition unit that acquires a response from a subject to a first question for estimating a skill level of the subject; a dialogue unit that, if the answer sentence does not satisfy a predetermined condition, acquires a second question sentence by inputting an instruction to generate a second question sentence different from the first question sentence into a generative model; and Equipped with The acquisition unit acquires a response sentence from the subject to the second question sentence, The dialogue unit is characterized in that, when the answer sentence to the second question sentence satisfies the condition, the skill level obtained by inputting the answer sentence together with an evaluation instruction for evaluating the skill level based on the answer sentence into a generative model is determined to be the final skill level of the subject.

[0007] In the information processing system of the present invention, The condition may be a condition regarding the credibility of a skill level obtained from the answer text.

[0008] In the information processing system of the present invention, The definition of the credit rating may be set or changed in response to a user operation.

[0009] In the information processing system of the present invention, The condition may be that the credibility obtained by inputting a credibility determination criterion together with the answer sentence into the generative model is equal to or greater than a threshold value.

[0010] In the information processing system of the present invention, The dialogue unit may acquire the second question sentence by inputting the generation instruction and a message indicating that the second question sentence is a question for obtaining a higher credibility to the generation model together with the answer sentence acquired by the acquisition unit.

[0011] In the information processing system of the present invention, The dialogue unit may generate the first question sentence by inputting the definition for each skill level into a generative model.

[0012] In the information processing system of the present invention, The definition for each skill level may be set or changed in response to a user operation.

[0013] In the information processing system of the present invention, The dialogue unit includes: inputting a presentation instruction for presenting a reason for evaluating the skill level into the generative model, thereby acquiring a reason for evaluating the skill level together with the skill level from the generative model; The determined skill level and the reason for the evaluation may be presented.

[0014] In the information processing system of the present invention, The dialogue unit includes: a presentation instruction for presenting a basis for the evaluation reason is input to the generative model, thereby further acquiring a part of the answer sentence that is the basis for the evaluation reason from the generative model; The skill level, the reason for the evaluation, and the basis for the evaluation may be presented.

[0015] In the information processing system of the present invention, The dialogue unit may further input features of the subject whose skill level is equal to or higher than a threshold into the generative model, thereby generating the first question sentence.

[0016] In the information processing system of the present invention, The subject is a job seeker, The skill level is a skill level for the recruitment content of a specified recruitment business, The dialogue unit may further generate the first question by inputting characteristics of a candidate accepted by the recruiting business into the generation model.

[0017] In the information processing system of the present invention, The subject is a job seeker, The skill level is a skill level for the recruitment content of a specified recruitment business, The dialogue unit may further generate the first question by inputting information about the recruiting provider into the generation model.

[0018] In the information processing system of the present invention, The subject is a job seeker, The dialogue unit may further generate the first question by inputting information about the job seeker into the generative model.

[0019] In the information processing system of the present invention, The first question may be a question for assessing the skill level for one item.

[0020] In the information processing system of the present invention, The first question may be a question for evaluating the skill level for each of a plurality of items.

[0021] The program of the present invention comprises: A program for causing a computer to function as an acquisition unit and a dialogue unit, The acquisition unit acquires a response from the subject to a first question for estimating a skill level of the subject, when the answer sentence does not satisfy a predetermined condition, the dialogue unit acquires a second question sentence by inputting an instruction to generate a second question sentence different from the first question sentence into a generation model; The acquisition unit acquires a response sentence from the subject to the second question sentence, The dialogue unit is characterized in that, when the answer sentence to the second question sentence satisfies the condition, the skill level obtained by inputting the answer sentence together with an evaluation instruction for evaluating the skill level based on the answer sentence into a generative model is determined to be the final skill level of the subject.

[0022] An information processing method performed by a computer having a control unit, The control unit acquires a response from the subject to a first question for estimating a skill level of the subject; a step of acquiring a second question sentence by inputting, into a generative model, an instruction to generate a second question sentence different from the first question sentence, when the answer sentence does not satisfy a predetermined condition, by the control unit; The control unit acquires a response from the subject to the second question; The method is characterized in that it includes a step in which, when the answer sentence to the second question sentence satisfies the condition, the control unit determines the skill level obtained by inputting, together with the answer sentence, an evaluation instruction for evaluating the skill level based on the answer sentence into a generative model as the final skill level of the subject. Effect of the Invention

[0023] According to the information processing system, program, and information processing method of the present invention, the accuracy of human resource evaluation using a generative model is improved. [Brief description of the drawings]

[0024] [Figure 1] This is an overall configuration diagram of the recruitment management system. [Diagram 2] FIG. 13 is a diagram illustrating an example of a data configuration of an item list. [Diagram 3] FIG. 2 is a diagram illustrating an example of a data configuration of creditworthiness information. [Figure 4] 13 is a flowchart showing a selection process. [Diagram 5] 13 is a flowchart showing a selection process. [Figure 6] 13 is a flowchart showing a selection process. [Figure 7] 13 is a flowchart showing a selection process. [Figure 8] 13 is a flowchart showing a selection process according to the second embodiment. [Figure 9] 13 is a flowchart showing an evaluation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] (First embodiment) Hereinafter, a first embodiment will be described with reference to the drawings. Fig. 1 is a diagram showing the overall configuration of a recruitment management system 1 according to this embodiment. The recruitment management system 1 is an information processing system that manages the recruitment selection of job seekers using a generative model.

[0026] The hiring management system 1 comprises a hiring management device 10, a job seeker terminal 20, a recruiting provider terminal 30, and a generative model device 40. The hiring management device 10, the job seeker terminal 20, the recruiting provider terminal 30, and the generative model device 40 are connected to each other so as to be able to communicate with each other via a communication network N such as the Internet.

[0027] The job seeker terminal 20 is a device used by a job seeker. The job seeker terminal 20 transmits job seeker information, which is information about a job seeker, such as application documents such as a resume and a curriculum vitae, to the recruitment management device 10. The recruiting provider terminal 30 is a device used by a person in charge of a recruiting provider. The recruiting provider terminal 30 transmits, for example, recruiting requirements and other information about human resources recruited by the job seeker, and accepted candidate information, which is information about the characteristics of accepted candidates, to the recruitment management device 10. The recruitment management device 10 performs processing related to the recruitment of a job seeker based on the recruitment information and job seeker information. The recruitment management device 10 is an example of an information processing system. The generative model device 40 stores a generative model 410. The recruitment management device 10 performs processing related to recruitment through dialogue with the generative model 410.

[0028] The hiring management device 10 is a computer or the like, and performs AI selection, which is the selection of job seekers using dialogue with a generative model 410 stored in a generative model device 40. The hiring management device 10 is configured with a computer or the like, and includes a control unit 100, a storage unit 110, a communication unit 120, and a user interface (UI) unit 130.

[0029] The control unit 100 includes a processor such as a CPU (Central Processing Unit) and controls the operation of the recruitment management device 10. The storage unit 110 includes, for example, a hard disk drive (HDD), a random access memory (RAM), a read only memory (ROM), and a solid state drive (SSD). Furthermore, the storage unit 110 is not limited to being built into the recruitment management device 10, and may be a storage medium (for example, a USB memory) that is detachably attached to the recruitment management device 10. In this embodiment, the storage unit 110 stores various programs executed by the control unit 100 and various data.

[0030] The communication unit 120 includes a communication interface for communicating with an external device wirelessly or via a wired connection. The control unit 100 transmits and receives data between the job seeker terminal 20, the recruiting provider terminal 30, and the generative model device 40 via the communication unit 120. The UI unit 130 includes an operation unit for accepting user operations, a display unit for displaying various information, a microphone for picking up speech and the like, and a speaker for emitting sound.

[0031] The job seeker terminal 20 includes a control unit 200, a storage unit 210, a communication unit 220, and a UI unit 230. The control unit 200, the storage unit 210, the communication unit 220, and the UI unit 230 are similar to the control unit 100, the storage unit 110, the communication unit 120, and the UI unit 130 of the recruitment management device 10, respectively.

[0032] The recruiting provider terminal 30 comprises a control unit 300, a storage unit 310, a communication unit 320, and a UI unit 330. The control unit 300, the storage unit 310, the communication unit 320, and the UI unit 330 are respectively similar to the control unit 100, the storage unit 110, the communication unit 120, and the UI unit 130 of the recruitment management device 10. The job seeker terminal 20 and the recruiting provider terminal 30 may be portable devices such as smartphones, PC tablets, etc.

[0033] The generative model device 40 is a computer or the like. The generative model device 40 stores a generative model 410 that receives a question as input and outputs a result. The generative model 410 is a learning model generated by machine learning, and generates a result according to a question (prompt) from a user who accesses via a communication network N. The generative model may be a Generative Pre-trained Transformer or a language model based thereon. Examples of the generative model 410 include ChatGPT, GPT-3, GPT-4 by OpenAI, and GPT-J by EleutherAI.

[0034] The generative model 410 learns from a huge amount of data and then makes predictions for new inputs. The generative model device 40 hosts the generative model 410, calls the generative model 410 in response to a request, and returns the result. When the generative model 410 receives a prompt, which is a command statement for giving an instruction from the hiring management device 10, it performs a calculation process to generate a result corresponding to the prompt. The generated result is then transmitted to the hiring management device 10.

[0035] Next, a configuration of the control unit 100 of the recruitment management device 10 will be described. The control unit 100 functions as an acquisition unit 101 and a dialogue unit 102 by executing a program stored in the storage unit 110. Note that, in the following, the processes described as being performed by the acquisition unit 101 and the dialogue unit 102 are processes performed by the control unit 100 by executing a program.

[0036] In the AI ​​selection process, the acquiring unit 101 acquires the job seeker's response to the question generated by the generative model 410 from the job seeker terminal 20 via the communication unit 120. The response transmitted from the job seeker terminal 20 is assumed to be text data. As another example, the response transmitted from the job seeker terminal 20 may be voice data. When the response transmitted from the job seeker terminal 20 is voice data, the control unit 100 converts the voice data into text data by voice recognition processing, and the acquiring unit 101 acquires the text data.

[0037] The dialogue unit 102 transmits a prompt, which is an instruction sentence for giving an instruction, to the generative model 410 via the communication unit 120. Then, the dialogue unit 102 acquires an answer to the prompt. For example, in response to a prompt instructing a job seeker to create a question for a job interview, the generative model 410 creates a question for the job interview, and the dialogue unit 102 acquires this question as an answer.

[0038] Next, the process of recruitment selection will be described. For example, at a recruitment agency, when a job seeker submits application documents in response to the recruitment requirements, the documents are screened, and as the next step, AI selection is carried out instead of an interview by a recruiter or as a selection process separate from the interview. In AI selection, a dialogue with the generative model 410 is used to confirm multiple items that have been set in advance. Note that the recruitment selection may include multiple processes.

[0039] The following describes AI selection using the generative model 410. In the AI ​​selection, the item list and credibility information are referenced. These pieces of information are stored in the storage unit 110.

[0040] FIG. 2 is a diagram showing an example of an item list. The item list is information showing a list of each item to be confirmed in AI selection, a definition of each item, and further showing the corresponding skill level and its definition. As shown in FIG. 2, in this embodiment, the items are classified into three hierarchies, from the top level, namely, major items, medium items, and minor items. Furthermore, multiple medium items are set for one major item. In the example of FIG. 2, three medium items, "(1-1) Emphasis on achievement," "(1-2) Elimination of uncertainty," and "(1-3) Initiative," are shown for the major item "(1) Results orientation."

[0041] Furthermore, multiple small items are set for one medium item. In the example of FIG. 2, three small items, "(A) Motivation and motivation to achieve," "(B) Scope of influence of actions to achieve," and "(C) Degree of innovation," are shown for "(1-1) Emphasis on achievement." Furthermore, definitions of each item are set for each of the large items, medium items, and small items. The number of hierarchies of these items, the number of items in each tier, and the contents of each item are set in advance by the person in charge of the recruiting business or the administrator of the recruitment management device 10. Moreover, the number of hierarchies of these items, the number of items in each tier, and the contents of each item are arbitrary and can be set freely.

[0042] In AI selection, a skill level is specified for each of the most detailed items (subitems in this embodiment). Here, the skill level is an evaluation value indicating the degree of ability for that item. For example, five skill levels from 1 to 5 are set for the subitem "(A) Motivation and drive to achieve" belonging to the medium item "(1-1) Emphasis on achievement". The higher the number of the skill level, the higher the skill level. Furthermore, a definition of the level is set in advance for each skill level. The definition of the skill level is set according to the experience of the job seeker. In this way, the skill level is judged according to the experience, so that a highly reliable skill level can be obtained. The number of stages of these skill levels and the definition of each skill level are set in advance by the administrator of the recruitment management device 10, the person in charge of the recruiting business, etc. Furthermore, the number of stages of these skill levels and the definition of each level are arbitrary and can be set freely.

[0043] In AI selection, the skill level of the job seeker for each small item is identified through dialogue with the job seeker, and then an evaluation is made of how trustworthy this skill level is. A trust index is used for this evaluation. The higher the trust index value, the more likely the skill level is to be trustworthy.

[0044] FIG. 3 is a diagram showing an example of the credibility information. The credibility information is information showing the credibility value and the definition (criteria) of each value. In the example of FIG. 3, the credibility is set to five levels from "Credit 1" to "Credit 5". The more specific the content is, such as the period of time spent working on a certain project, the results, etc., the higher the credibility is set. A definition is set in advance for each credibility value. By setting the credibility definition in this way, it is possible to clarify what judgment criteria the credibility value corresponds to. The number of credibility values ​​(number of levels) and the definition are set in advance by the administrator of the recruitment management device 10, the person in charge of the recruiting business, etc. In addition, the number of levels and the definition of the credibility value are arbitrary and can be set freely.

[0045] 4 to 7 are flowcharts showing the selection process related to AI selection. First, the dialogue unit 102 provides preconditions to the generation model 410 (step S100). Here, the preconditions include an item list, credibility information, information on the target recruitment business, characteristics of the job-seekers at the recruitment business, and information on the job seekers. The information on the recruitment business is information on the recruitment business, and more specifically, is information that can be referred to in matching with the applicant's requests in recruitment. The information on the recruitment business includes the recruitment business's philosophy, culture, business content, corporate information, recruitment requirements, etc. The characteristics of the job-seekers may be information indicating the characteristics of applicants who have already been offered a job, or may be information indicating the characteristics of job-seekers that the recruitment business is looking for, that is, information indicating the characteristics of the ideal job-seekers. The information on the job seekers is information on the resumes and curriculum vitae of the job seekers. The dialogue unit 102 instructs the generation model 410 to carry out recruitment selection based on the preconditions and these.

[0046] Next, the dialogue unit 102 selects a minor item to be processed (step S102). Specifically, the dialogue unit 102 selects one of the major items as the processing target, and further selects the first medium item included in the major item to be processed as the processing target. Then, the dialogue unit 102 selects the first minor item included in the medium item to be processed as the processing target. Note that, in the following, an example will be described in which the major item "(1) Results orientation", the medium item "(1-1) Achievement emphasis", and the minor item "(A) Motivation and motivation to achieve" are selected as the processing targets.

[0047] Next, the dialogue unit 102 acquires from the generative model 410 a first question sentence corresponding to the small item selected as the processing target (step S104). Here, the first question sentence is a question sentence for estimating the skill level of the job seeker and its credibility for the small item selected as the processing target. Specifically, the dialogue unit 102 requests the generative model 410 to generate a first question sentence by transmitting a prompt to the generative model device 40 to instruct generation of a first question sentence corresponding to one small item selected as the processing target. When the generative model device 40 receives the prompt, it inputs it to the generative model 410. As a result, the generative model 410 generates the first question sentence. The generative model device 40 transmits the first question sentence to the recruitment management device 10. Then, the dialogue unit 102 acquires the first question sentence as a response to the transmitted prompt. For example, the following sentence is obtained as the first question sentence. "Tell me about a time in your work experience when you took the initiative in planning or adjusting your actions to achieve a goal. Describe in detail the challenges you faced, the specific steps you took, and the results."

[0048] Next, the dialogue unit 102 presents the initial question to the job seeker (step S106). Specifically, the dialogue unit 102 transmits the initial question to the job seeker terminal 20 used by the job seeker via the communication unit 120. In the job seeker terminal 20, the control unit 200 acquires the initial question and displays it on the display unit of the UI unit 230. This allows the job seeker to check the initial question generated by the generative model 410. After checking the initial question, the job seeker inputs a response to it using the operation unit of the job seeker terminal 20. This response is transmitted from the job seeker terminal 20 to the recruitment management device 10. For example, the following sentence is obtained as the response: "I have a clear standard for work that requires delivery one or two days before the due date at the very least. So I break down tasks into smaller chunks and create a schedule with the goal of completing them one or two days before the deadline. This helps me to notice any missed tasks before delivery, and by submitting work early, I can gain more credibility with my clients."

[0049] Then, in the recruitment management device 10, the control unit 100 acquires the reply from the job seeker via the communication unit 120 (step S108). The control unit 100 displays the reply on the display unit of the UI unit 130. As another example, the control unit 100 may transmit the reply to the recruiting provider terminal 30 and cause it to be displayed on the display unit of the recruiting provider terminal 30.

[0050] Next, the dialogue unit 102 acquires the skill level and its credibility according to the answer sentence from the generative model 410 (step S110). Specifically, the dialogue unit 102 requests an evaluation of the skill level and credibility based on the answer sentence by transmitting an evaluation prompt to the generative model device 40 along with the answer sentence. When the generative model device 40 receives the prompt, it inputs it to the generative model 410. As a result, the generative model 410 calculates the skill level and credibility according to the prompt and outputs them. The skill level and credibility are transmitted to the recruitment management device 10. Then, in the recruitment management device 10, the dialogue unit 102 acquires the skill level and credibility as the evaluation results. For example, a skill level of 3 and a credibility of 4 are obtained for the previous example answer sentence.

[0051] Next, the dialogue unit 102 presents the obtained skill level and trustworthiness (S112). Specifically, the dialogue unit 102 displays the skill level and trustworthiness on the display unit of the UI unit 130 of the recruitment management device 10. This allows the user of the recruitment management device 10 to confirm the skill level and trustworthiness.

[0052] As another example, the skill level and the trustworthiness may be presented to the user of the recruiting provider terminal 30. In this case, the dialogue unit 102 transmits the skill level and the trustworthiness to the recruiting provider terminal 30 via the communication unit 120. Then, in the recruiting provider terminal 30, the control unit 300 acquires the skill level and the trustworthiness and displays them on the display unit of the UI unit 330. This allows the recruiting provider to check the skill level and the trustworthiness as well.

[0053] Next, the dialogue unit 102 checks whether the answer sentence is negative (step S114). The answer sentence is negative when it contains a sentence that denies the content of the question sentence, such as "We kept the delivery date, but we did not make a plan for that." In the case of such a sentence in the negative form, it is highly likely that the requested content is not included, and it is determined that evaluation cannot be performed from this answer sentence. It is assumed that whether the answer sentence is negative is determined by the generation model 410. Specifically, the prompt input to the generation model 410 includes a command to output information to that effect if the answer sentence is negative. If the answer sentence is negative (Y in step S114), the dialogue unit 102 advances the process to step S140.

[0054] If the reply sentence is not in the negative form (N in step S114), the dialogue unit 102 checks the credibility of the skill level obtained for the reply sentence (step S116). If the credibility is equal to or greater than a preset threshold (Y in step S116), the dialogue unit 102 advances the process to step S120 shown in Fig. 5. It is assumed that the threshold is preset. For example, credibility 3 is set as the threshold.

[0055] If the reliability is equal to or greater than the threshold, the skill level obtained for the reply sentence is treated as a reliable value in the processes from step S120 onward. In step S120, the dialogue unit 102 checks whether or not a provisional skill level has been set. Here, the provisional skill level is a skill level that is provisionally set before the skill level is finalized. In the initial state, no value is set for the provisional skill level.

[0056] If the provisional skill level has not been set (N in step S120), the dialogue unit 102 advances the process to step S124. If the provisional skill level has been set (Y in step S120), the dialogue unit 102 compares the skill level acquired from the generative model 410 in the most recent step S110 (acquired skill level) with the provisional skill level (step S122). If the acquired skill level is a higher value than the provisional skill level (Y in step S122), the dialogue unit 102 sets the acquired skill level to the provisional skill level (step S124). If the provisional skill level is in the initial state and the acquired skill level is skill level 3, the provisional skill level is set to 3.

[0057] If the acquired skill level is the same as the provisional skill level or lower than the provisional skill level (N in step S122), the dialogue unit 102 does not update the provisional skill level. Specifically, in this case, the dialogue unit 102 advances the process to step S140. Note that although the provisional skill level may be updated in the selection process, the provisional skill level once set is not updated to a lower level. This is to prevent the loop process from being repeated, thereby preventing the amount of calculation from becoming too large.

[0058] After the process of step S124, the dialogue unit 102 checks whether or not a skill level higher than the provisional skill level is set for the minor item being processed (step S126). For example, for the minor item "(A) Motivation and motivation to achieve", skill levels 1 to 5 are set. Therefore, if a value of 4 or less is set for the provisional skill level, it is determined that a skill level higher than the provisional skill level exists.

[0059] If there is a skill level higher than the provisional skill level (Y in step S126), the dialogue unit 102 acquires a level-up question from the generative model 410 (step S128). Here, the level-up question is a question for confirming whether the job seeker has a higher skill level. Specifically, the dialogue unit 102 requests the generative model 410 to generate a level-up question by sending a prompt to instruct the job seeker to generate a level-up question. When the generative model device 40 receives the prompt, it inputs it to the generative model 410. The generative model 410 generates a level-up question based on the prompt and the information obtained up to that point.

[0060] The generative model device 40 transmits the level-up question generated by the generative model 410 to the recruitment management device 10. Then, the dialogue unit 102 acquires the level-up question as a response to the transmitted prompt.

[0061] For example, when the provisional skill level is set to "3", the following sentence is obtained as the level-up question. "Please tell us about an example of an efficiency improvement idea that you proposed or implemented that helped the organization save time or reduce costs. Also tell us what results you were able to achieve."

[0062] Next, the dialogue unit 102 presents the level-up question to the job seeker (step S130). Next, the dialogue unit 102 resets the probing question count to zero (step S132). The probing question count will be described later.

[0063] The control unit 100 then advances the process to step S108. After step S108, the control unit 100 advances the process according to the answer sentence input by the job seeker. Assume that the answer sentence acquired in step S108 is not negative, has a reliability equal to or greater than a threshold, and has a skill level higher than the provisional skill level. In this case, in step S124, the provisional skill level is updated to a higher level. In this manner, the processes from step S108 to step S132 are repeated, thereby updating the provisional skill level to a higher level.

[0064] Furthermore, in step S122, if the skill level is equal to or lower than the provisional skill level (N in step S122), the dialogue unit 102 advances the process to step S140 shown in Fig. 6. In step S140, the dialogue unit 102 sets the value set in the provisional skill level at the time of processing as the final skill level. Furthermore, in the case where the provisional skill level has not been set at the time of processing, the dialogue unit 102 sets information indicating that evaluation is impossible in the final skill level.

[0065] 5, if there is no skill level higher than the provisional skill level (N in step S126), the dialogue unit 102 advances the process to step S140. In this case, the dialogue unit 102 similarly sets the value set as the provisional skill level at the time of processing as the final skill level in step S140. That is, in this case, the dialogue unit 102 sets the maximum value of the skill levels for the minor items being processed as the final skill level. Since the maximum skill level for "(A) Motivation and motivation to achieve" is "5", the final skill level is set to "5".

[0066] 4, if the reply sentence is negative (Y in step S114), the dialogue unit 102 advances the process to step S140. In this case, as in the above, the dialogue unit 102 sets the value set as the provisional skill level at the time of processing as the final skill level in step S140, or sets information indicating that evaluation is impossible. In this way, if the reply sentence is negative, the dialogue unit 102 ends checking the skill level at this point.

[0067] After the process of step S140, the dialogue unit 102 presents the final skill level (S142). Specifically, the dialogue unit 102 displays the final skill level on the display unit of the UI unit 130. Note that, as another example, the dialogue unit 102 may display the final skill level on the display unit of the UI unit 330 of the recruiting provider terminal 30.

[0068] The dialogue unit 102 checks whether there is a next small item (step S144). For example, when the small item to be processed is "(A) Motivation and motivation to achieve," it is determined that there is a next small item because the other small items in the medium item "(1-1) Emphasis on achievement" have not been selected.

[0069] If there is a next subitem (Y in step S144), the dialogue unit 102 selects the next subitem as the processing target (step S146). Next, the dialogue unit 102 resets the value of the probing question count to zero (step S148), and then proceeds to the process at step S104. In this case, the dialogue unit 104 continues the process on the new subitem that has become the processing target from step S104 onward.

[0070] In step S142, if there is no next small item (N in step S144), the dialogue unit 102 assumes that the processing has been completed for all small items belonging to the medium item being processed, and ends the processing. Note that this processing is executed for all medium items belonging to all large items included in the item list.

[0071] On the other hand, if the credibility of the skill level obtained for the answer obtained in step S110 is less than the threshold value (N in step S116), the skill level cannot be determined as it is. In this case, the dialogue unit 102 presents a further question to the job seeker in the process from step S150 onwards shown in FIG. 7. Here, the further question is a question for further probing the content of the answer of the job seeker, and hereinafter, this question is referred to as a probing question. A probing question is a question that will result in a more credible skill level than the previous answer. It is preferable that the probing question is a question that will make the job seeker answer about his or her actual experience.

[0072] Specifically, in step S116, if the trustworthiness is less than the threshold value or if the trustworthiness is unassessable (N in step S116), the dialogue unit 102 advances the process to step S150 shown in Fig. 7. In step S150, the dialogue unit 102 checks the probing question count. The probing question count is set to zero in the initial state, and is counted up every time a probing question is generated. In addition, a maximum count value that is the maximum value of the probing question count is preset.

[0073] If the probing question count is less than the maximum count value (Y in step S150), the dialogue unit 102 acquires a probing question from the generative model 410 (step S152). Specifically, the dialogue unit 102 requests the generative model 410 to generate a probing question by sending a prompt to instruct the job seeker to generate a probing question. The prompt includes information that the probing question is a question for obtaining a skill level with a higher degree of reliability than answers obtained so far. More preferably, the prompt includes information that the probing question is a question that prompts the job seeker to answer based on his or her actual experience.

[0074] When the generative model device 40 receives the prompt, it inputs it to the generative model 410. The generative model 410 generates a probing question based on the prompt and the information obtained up to that point. The probing question is then transmitted to the recruitment management device 10.

[0075] In response to this, the dialogue unit 102 acquires a probing question as a response to the prompt. Next, the dialogue unit 102 presents the probing question to the job seeker (step S154).

[0076] For example, suppose the following response was received in response to the initial question: "I made a schedule and broke down the tasks into small pieces." In this case, the credibility is evaluated as 2, and it is determined that a probing question is necessary. In this case, for example, the following probing question is obtained. "When you created a schedule and broke down your tasks, what specific actions did you take based on that plan? What results did you achieve? Please also tell us about the goals and standards you set for yourself."

[0077] Then, the dialogue unit 102 increments the probing question count by 1 (step S156). The control unit 100 then advances the process to step S108. After step S108, the control unit 100 advances the process according to the answer sentence input by the job seeker.

[0078] Also, in step S150, if the probing question count is equal to or greater than the maximum count value (N in step S150), the process proceeds to step S140, where the skill level is determined.

[0079] In this way, when the credibility does not meet the threshold, the dialogue unit 102 automatically generates probing questions and presents them to the job seeker, thereby obtaining further answers from the job seeker. Therefore, the skill level and credibility can be evaluated again based on the newly obtained answers.

[0080] As described above, in the hiring management system 1 of this embodiment, if the reliability of the answer obtained in response to one question is low, the hiring management device 10 generates a probing question as a second question and allows the job seeker to input the answer again. This makes it possible to obtain an answer with a higher reliability. Furthermore, the hiring management device 10 presents a new question corresponding to the skill level as the second question depending on the content of the answer, thereby making it possible to obtain information from the job seeker required to determine the skill level. In this way, the hiring management system 1 of this embodiment makes it possible to improve the accuracy of human resource evaluation in hiring selection using a generative model.

[0081] In this embodiment, a definition is set for each skill level, so that when a skill level is presented, the user can refer to the definition to understand its meaning. Therefore, more detailed information can be obtained than if a numerical value were simply presented. Furthermore, a definition is set for each skill level so that the skill level is determined based on the experience of the job seeker. Therefore, a more reliable skill level can be obtained.

[0082] Second embodiment Next, the hiring management system 1 according to the second embodiment will be described, focusing mainly on the differences from the first embodiment. In the second embodiment, the dialogue unit 102 of the hiring management device 10 causes the generative model 410 to generate, as an initial question, a question capable of evaluating the skill level and the corresponding credibility for each of multiple small items belonging to one medium item.

[0083] 8 is a flowchart showing a selection process according to the second embodiment. First, the dialogue unit 102 provides a precondition to the generative model 410 (step S200). This process is similar to the process of step S100 described with reference to FIG.

[0084] Next, the dialogue unit 012 selects a medium item to be processed (step S202). Specifically, the dialogue unit 102 selects one of the main items as a processing target, and further selects the first medium item included in the main item to be processed as a processing target. Note that, in the following, an example will be described in which the main item "(1) Results-oriented" and the medium item "(1-1) Achievement-oriented" are selected as processing targets.

[0085] Next, the dialogue unit 102 acquires an initial question for the medium item selected as the processing target from the generative model (step S204). Specifically, the dialogue unit 102 requests the generation of an initial question to the generative model 410 by sending a prompt to instruct generation of an initial question corresponding to the selected medium item. This prompt includes information specifying that the initial question evaluates the skill levels and the credibility of all the subitems included in the processing target medium item.

[0086] When the generative model device 40 receives the prompt, it inputs it to the generative model 410. As a result, an initial question sentence is generated by the generative model. The generative model device 40 transmits the initial question sentence to the recruitment management device 10. Then, the dialogue unit 102 acquires the initial question sentence as a response to the transmitted prompt. For example, the following sentence is obtained as the initial question sentence. "Tell me about a specific challenge you faced in your previous job. Please tell me in detail how you addressed the challenge, what actions you took, and what results your actions produced."

[0087] Next, the dialogue unit 102 presents the initial question to the job seeker (step S206). As a result, a response is input from the job seeker terminal 20, and the response is transmitted to the recruitment management device 10. In the recruitment management device 10, the control unit 100 acquires the response from the job seeker via the communication unit 120 (step S208). For example, the following answer may be given to the above question: "We had a project where we had to deliver several websites in a very short period of time. So we came up with the idea of ​​creating a CMC (Contents Management System) to mass-produce websites..."

[0088] Next, the dialogue unit 102 acquires the skill level and its credibility according to the answer sentence from the generation model 410 (step S210). Note that here, the dialogue unit 102 acquires the skill levels and their credibility for all small items belonging to the medium item to be processed.

[0089] Specifically, the dialogue unit 102 requests the generative model 410 to evaluate the skill levels and trustworthiness of all minor items belonging to the medium item being processed based on the answer sentence, together with the answer sentence. When the generative model device 40 receives the prompt, it inputs it to the generative model 410. In response to the prompt, the generative model 410 calculates the skill levels and trustworthiness of all minor items belonging to the medium item being processed, and outputs them. The skill levels and trustworthiness are transmitted to the recruitment management device 10, and the dialogue unit 102 obtains the skill level and trustworthiness of each minor item as the evaluation result.

[0090] For example, the following evaluation results are obtained for the above response: (A) Motivation and drive to achieve: Skill level 4, Trust level 4 (B) Scope of impact of actions on achievement: Skill level 3, Trust level 4 …

[0091] In this way, in the second embodiment, the dialogue unit 102 can obtain the skill levels and the confidence levels for all the small items belonging to the medium item being processed from one reply sentence from the job seeker.

[0092] Next, the dialogue unit 102 presents the obtained skill level and credibility (S212). Next, the dialogue unit 102 executes an evaluation process for all small items to be processed (step S214). FIG. 9 is a flowchart showing detailed processing in the evaluation process (step S214). The evaluation process (step S214) is executed for each small item to be processed. In the evaluation process (step S214), the dialogue unit 102 first checks whether the answer sentence is in the negative form (step S230). If the answer sentence is in the negative form (Y in step S230), the dialogue unit 102 advances the process to step S232. In step S232, the dialogue unit 102 sets the value set in the provisional skill level at the time of processing as the final skill level. Also, if the provisional skill level is not set at the time of processing, the dialogue unit 102 sets information indicating that evaluation is not possible in the final skill level. With this, the dialogue unit 102 ends the evaluation process.

[0093] On the other hand, if the answer sentence is not negative (N in step S230), the dialogue unit 102 checks the credibility of the skill level obtained for the answer sentence (step S234). If the credibility is less than the threshold (N in step S234), the dialogue unit 102 determines that a probing question is necessary (step S236) and ends the evaluation process.

[0094] If the reliability is equal to or greater than a preset threshold (Y in step S234), the dialogue unit 102 checks whether or not a provisional skill level has been set (step S238). If the provisional skill level has not been set (N in step S238), the dialogue unit 102 advances the process to step S244. If the provisional skill level has been set (Y in step S238), the dialogue unit 102 compares the skill level acquired from the generative model 410 (acquired skill level) with the provisional skill level in 210 (step S240).

[0095] If the acquired skill level is equal to or lower than the provisional skill level (N in step S240), the dialogue unit 102 advances the process to step S242. In step S242, the dialogue unit 102 sets the value set in the provisional skill level at the time of processing as the final skill level. Furthermore, if the provisional skill level has not been set at the time of processing, the dialogue unit 102 sets information indicating that evaluation is not possible in the final skill level. This completes the evaluation process.

[0096] If the acquired skill level is a higher value than the provisional skill level (Y in step S240), the dialogue unit 102 sets the acquired skill level to the provisional skill level (step S244). Next, the dialogue unit 102 checks whether or not there is a skill level higher than the provisional skill level (step S246). If there is no skill level higher than the provisional skill level (N in step S246), the dialogue unit 102 ends the evaluation process. If there is a skill level higher than the provisional skill level (Y in step S246), the dialogue unit 102 determines that a level-up question is necessary (step S248), and ends the evaluation process.

[0097] 8, after the evaluation process (step S214) is performed, the dialogue unit 102 judges whether or not the evaluation of all the small items belonging to the medium item being processed is completed (step S216). When the final skill level is determined for all the small items, it is judged that the evaluation is completed.

[0098] When the dialogue unit 102 has finished evaluating all the sub-items (Y in step S216), it advances the process to step S218. When the dialogue unit 102 determines that a probing question is necessary (probing question necessary in step S216), it obtains a probing question sentence from the generative model 410 and presents this to the job seeker (step S220), and then advances the process to step S208. Note that this process is similar to the processes in steps S152 and S154. In the processes from step S208 onwards, the dialogue unit 102 obtains a response sentence from the job seeker and again evaluates the skill level and credibility based on this.

[0099] Furthermore, if the dialogue unit 102 determines that a level-up question is necessary (Level-up Question Required in step S216), it obtains a level-up question sentence from the generation model 410, presents this to the job seeker (step S222), and then proceeds to step S208. This process is similar to the processes in steps S128 and S130. In the processes from step S208 onwards, the dialogue unit 102 obtains a response sentence from the job seeker, and again evaluates the skill level and credibility based on this. Note that if the dialogue unit 102 determines that a level-up question is necessary and also that a probing question is necessary, the probing question is given priority.

[0100] In step S218, the dialogue unit 102 checks whether or not there is a next medium item that has not been selected as a processing target. If there is a next medium item (Y in step S218), the dialogue unit 102 advances the process to step S202. In this case, in step S202, the dialogue unit 102 selects the unselected medium item as a processing target and performs the subsequent processes. If there is no next medium item (N in step S218), the dialogue unit 102 ends the selection process.

[0101] The hiring management system 1 of the second embodiment can also improve the accuracy of hiring selection using a generative model. Furthermore, in this embodiment, the hiring management device 10 can obtain the skill level and credibility corresponding to each of multiple small items from a single response text, thereby making the selection process more efficient.

[0102] The above embodiment is one example for carrying out the present invention, and various other embodiments can be adopted. For example, various modifications and changes are possible within the scope of the gist of the present invention described in the claims, such as applying one modified example to another modified example. For example, some of the configurations of the above embodiment may be omitted, and the order of processing may be changed or omitted.

[0103] In the first modified example, when the evaluation result of the answer sentence input by the job seeker does not satisfy a predetermined condition, the dialogue unit 102 may input an instruction to generate a question sentence different from the previous question sentence to the generation model 410, and the specific process for this is not limited to that described in the embodiment. For example, when the information that was previously set as a desired acquisition is not obtained as the evaluation result of the answer sentence, the dialogue unit 102 may input an instruction to generate a new question sentence to the generation model 410 in order to obtain the information.

[0104] A second modified example will now be described. In this embodiment, information such as questions presented by the dialogue unit 102 is displayed on the job seeker's display unit, but the output form of this information is not limited to the embodiment. As another example, the information may be output as voice by a speaker.

[0105] As a third modified example, the dialogue unit 102 may present, in step S112, the evaluation reason for obtaining such an evaluation result and its basis, together with the skill level and the credibility. Here, the basis is the part of the answer sentence that is used as the basis for the judgment. In this case, the dialogue unit 102 inputs the evaluation reason and a prompt for instructing the presentation of the basis to the generative model 410. This allows the dialogue unit 102 to obtain not only the skill level and the credibility, but also the evaluation reason and basis.

[0106] For example, the following sentences can be obtained as the evaluation reasons and grounds. The following sentences are the evaluation reasons corresponding to skill level 1 and credibility level 2. Reason for evaluation: "The reasons and background for specific actions are not fully explained, but the actions themselves are described." Reason: "I made a schedule and broke down the tasks into small pieces."

[0107] Similarly, the dialogue unit 102 may present not only the final skill level but also the corresponding credibility, evaluation reason and basis in step S142. In this case, in step S124, not only the acquired skill level but also the credibility, evaluation reason and basis corresponding to this acquired skill level are set together with the final skill level.

[0108] Also, in step S212, the dialogue unit 102 may present the reason for the evaluation and its basis, together with the skill level and the credibility.

[0109] As a fourth modified example, in this embodiment, the process of determining the skill level of a job seeker in AI selection has been described, but the target is not limited to a job seeker. The target may be any human resource whose skill level needs to be evaluated. For example, in cases where employees are evaluated within a company, an information processing system may be used to evaluate the skill level of each employee. Furthermore, in this case, instead of job offer information, the characteristics of a person (target) whose skill level is above a threshold value are included in the prerequisites.

[0110] According to the information processing system, program, and information processing method of the present embodiment configured as described above, the system includes an acquisition unit 101 that acquires an answer from the subject to a first question for estimating the skill level of the subject, and a dialogue unit 102 that acquires a second question by inputting an instruction to generate a second question different from the first question to the generation model when the answer does not satisfy a predetermined condition, and the acquisition unit 101 acquires an answer from the subject to the second question, and the dialogue unit 102 determines the skill level obtained by inputting an evaluation instruction to evaluate the skill level based on the answer to the generation model 410 together with the answer when the answer to the second question satisfies the condition as the final skill level of the subject. In this way, the second question is automatically generated and a corresponding answer is obtained, so that the accuracy of the employment selection using the generation model can be improved.

[0111] In the information processing system, the program, and the information processing method of the present embodiment, the condition may be a condition related to the reliability of the skill level obtained from the answer text. This makes it possible to determine whether or not to generate a second question text depending on the reliability of the skill level.

[0112] In the information processing system, the program, and the information processing method according to the present embodiment, the definition of the trustworthiness may be set or changed in response to a user operation. This allows the definition of the trustworthiness to be set or changed as desired.

[0113] In the information processing system, the program, and the information processing method of the present embodiment, the condition may be that the credibility obtained by inputting the credibility judgment criteria together with the answer sentence into the generation model is equal to or greater than a threshold value. This allows the credibility of the skill level to be automatically judged.

[0114] In the information processing system, the program, and the information processing method of the present embodiment, the dialogue unit 102 may acquire the second question sentence by inputting a generation instruction and a message to the effect that the second question sentence is a question for obtaining a higher credibility, together with the answer sentence acquired by the acquisition unit 101, to the generation model 410. In this way, the second question sentence can be acquired automatically.

[0115] In the information processing system, the program, and the information processing method of the present embodiment, the dialogue unit 102 may generate the first question by inputting the definition of each skill level into the generative model. This makes it possible to automatically obtain the first question.

[0116] In the information processing system, the program, and the information processing method of the present embodiment, the definition of each skill level may be set or changed in response to a user operation. This allows the definition of each skill level to be set arbitrarily.

[0117] In the information processing system, program, and information processing method of the present embodiment, the dialogue unit 102 may input a presentation instruction to present the evaluation reason of the skill level to the generative model 410, thereby acquiring the evaluation reason of the skill level from the generative model 410 together with the skill level, and presenting the determined skill level and the evaluation reason. This allows the user to confirm not only the skill level but also the evaluation reason.

[0118] In the information processing system, program, and information processing method of the present embodiment, the dialogue unit 102 may input a presentation instruction to present the grounds for the evaluation reason to the generative model 410, thereby further acquiring the part of the answer sentence that is the grounds for the evaluation reason from the generative model 410, and present the skill level, the evaluation reason, and the grounds. This allows the user to confirm not only the skill level, but also the evaluation reason and the grounds.

[0119] In the information processing system, the program, and the information processing method of the present embodiment, the dialogue unit 102 may further input the characteristics of a subject whose skill level is equal to or higher than a threshold into the generative model to generate the first question sentence. This makes it possible to obtain the first question sentence reflecting the characteristics of the subject.

[0120] In the information processing system, program, and information processing method of the present embodiment, the subject is a job seeker, the skill level is a skill level for the recruitment content of a specific recruiting business, and the dialogue unit 102 may further input the characteristics of the successful candidate to the recruiting business into the generation model to generate the first question. This makes it possible to obtain the first question text that reflects the characteristics of the successful candidate.

[0121] Furthermore, in the information processing system, program, and information processing method of this embodiment, the target person is a job seeker, the skill level is a skill level for the recruitment content of a specific recruiting business, and the dialogue unit 102 may generate the first question by further inputting information on the recruiting business into the generation model 410. In this way, the first question text reflecting the information on the recruiting business can be obtained.

[0122] In the information processing system, program, and information processing method of the present embodiment, the subject is a job seeker, and the dialogue unit 102 may further input information about the job seeker into the generative model to generate the first question. This makes it possible to obtain a first question sentence that reflects the information about the job seeker.

[0123] In the information processing system, the program, and the information processing method of the present embodiment, the first question may be a question for evaluating the skill level for one item, thereby obtaining a first question for evaluating the skill level for each item.

[0124] In the information processing system, the program, and the information processing method of the present embodiment, the first question may be a question for evaluating a skill level for each of a plurality of items, thereby obtaining the first question for evaluating a skill level for each of a plurality of items. [Explanation of symbols]

[0125] 1. Recruitment management system 10 Recruitment management device 20 Job Seeker Terminal 30 Recruitment business terminal 40 Generative Model Device 100 Control section 101 Acquisition Department 102 Dialogue Section 110 Storage section 120 Communications Department 130 UI section 200 Control section 210 Storage section 220 Communications Department 230 UI section 300 Control section 310 Storage section 320 Communications Department 330 UI section 410 Generative Model

Claims

1. an acquisition unit that acquires a response from a subject to a first question for estimating a skill level of the subject; a dialogue unit that, if the answer sentence does not satisfy a predetermined condition, acquires a second question sentence by inputting an instruction to generate a second question sentence different from the first question sentence into a generation model; and Equipped with The acquisition unit acquires a response sentence from the subject to the second question sentence, when the answer sentence to the second question sentence satisfies the condition, the dialogue unit determines the skill level obtained by inputting, together with the answer sentence, an evaluation instruction for evaluating a skill level based on the answer sentence into a generative model as a final skill level of the subject; The second question is a question for identifying the skill level. Information processing system.

2. The information processing system according to claim 1 , wherein the condition is a condition related to the reliability of a skill level obtained from the answer text.

3. The information processing system according to claim 2 , wherein the definition of the trustworthiness can be set or changed in response to a user operation.

4. The information processing system according to claim 2 , wherein the condition is that the credibility obtained by inputting a criterion for determining the credibility together with the answer sentence into the generation model is equal to or greater than a threshold value.

5. 5. The information processing system of claim 4, wherein the dialogue unit acquires the second question sentence by inputting the generation instruction and a message indicating that the second question sentence is a question for obtaining a higher credibility to the generation model together with the answer sentence acquired by the acquisition unit.

6. The information processing system according to claim 1 , wherein the dialogue unit generates the first question sentence by inputting the definition for each skill level into a generative model.

7. The information processing system according to claim 6 , wherein the definition for each skill level can be set or changed in response to a user operation.

8. The dialogue unit includes: inputting a presentation instruction for presenting a reason for evaluating the skill level into the generative model, thereby obtaining a reason for evaluating the skill level together with the skill level from the generative model; Presenting the determined skill level and the reason for the evaluation 7. The information processing system according to claim 6.

9. The dialogue unit includes: a presentation instruction for presenting a basis for the evaluation reason is input to the generative model, thereby further acquiring a part of the answer sentence that is the basis for the evaluation reason from the generative model; The information processing system according to claim 8 , further comprising: displaying the skill level, the reason for the evaluation, and the basis for the evaluation.

10. The information processing system according to claim 6 , wherein the dialogue unit further generates the first question sentence by inputting features of the subject whose skill level is equal to or higher than a threshold into the generation model.

11. The subject is a job seeker, The skill level is a skill level for the recruitment content of a specified recruitment business, The information processing system according to claim 6 , wherein the dialogue unit further generates the first question sentence by inputting characteristics of candidates accepted by the recruiting business into the generation model.

12. The subject is a job seeker, The skill level is a skill level for the recruitment content of a specified recruitment business, The information processing system according to claim 6 , wherein the dialogue unit further generates the first question sentence by inputting information about the recruiting business into the generation model.

13. The information processing system according to claim 6 , wherein the dialogue unit generates the first question sentence by further inputting information about the subject to the generative model.

14. The information processing system according to claim 1 , wherein the first question is a question for evaluating the skill level for one item.

15. The information processing system according to claim 1 , wherein the first question is a question for evaluating the skill level for each of a plurality of items.

16. A program for causing a computer to function as an acquisition unit and a dialogue unit, The acquisition unit acquires a response from the subject to a first question for estimating a skill level of the subject, when the answer sentence does not satisfy a predetermined condition, the dialogue unit acquires a second question sentence by inputting an instruction to generate a second question sentence different from the first question sentence into a generation model; The acquisition unit acquires a response sentence from the subject to the second question sentence, when the answer sentence to the second question sentence satisfies the condition, the dialogue unit determines the skill level obtained by inputting, together with the answer sentence, an evaluation instruction for evaluating a skill level based on the answer sentence into a generative model as a final skill level of the subject; The second question is a question for identifying the skill level. program.

17. An information processing method performed by a computer having a control unit, The control unit acquires a response from the subject to a first question for estimating a skill level of the subject; a step of acquiring a second question sentence by inputting, into a generative model, an instruction to generate a second question sentence different from the first question sentence, when the answer sentence does not satisfy a predetermined condition, by the control unit; The control unit acquires a response sentence from the subject to the second question sentence; and when the answer sentence to the second question sentence satisfies the condition, the control unit determines the skill level obtained by inputting, together with the answer sentence, an evaluation instruction for evaluating a skill level based on the answer sentence into a generative model as a final skill level of the subject. An information processing method, wherein the second question is a question for identifying the skill level.

Images