Skill evaluation device, skill evaluation method, and program

Abstract

This invention provides a skill evaluation device that utilizes the text analysis capabilities of an AI model while enabling appropriate evaluations similar to those performed by human evaluators. [Solution] A skill evaluation device that determines a person's skills by having an AI model evaluate a target text, comprising: an input unit that receives input of the target text from the person; a control unit that controls the execution of inference processing by the AI ​​model; and an inference unit that executes the inference processing by the AI ​​model, wherein the evaluation process by the AI ​​model includes a first phase that measures the quality of the description of the target text and a second phase that measures the person's skills, and the control unit controls the device so as not to proceed to the second phase if, as a result of the measurement by the AI ​​model in the first phase, it is determined that the description of the target text does not meet a predetermined quality standard.

Description

Technical Field

[0001] The present invention relates to a skill evaluation apparatus, a skill evaluation method, and a program using an AI model.

Background Art

[0002] Systems that use generative AI to evaluate the professional skills and practical experience of evaluation subjects based on texts such as resumes are known.

[0003] For example, Patent Document 1 describes a system in which an AI algorithm analyzes a resume and a skill sheet uploaded by a user, provides a candidate list of talent matching a job opening provided by a company, generates a skill check test corresponding to the job opening and transmits it to the user, and provides the user's test result to the company.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0005] It is also known that AI-based evaluation has problems such as a tendency to evaluate the overall evaluation being influenced by distinctive impressions (the halo effect) and a tendency to give higher evaluations to longer texts (redundancy bias). In addition, in the case of AI-based evaluation, when the content described in a resume is insufficient, unlike a human evaluator, it has not been possible to ask questions to the evaluation subject to supplement judgment materials and thereby perform an accurate evaluation.

[0006] The present invention has been made in consideration of the circumstances described above, and an object of the present invention is to provide a skill evaluation apparatus capable of performing appropriate evaluation similar to that performed by a human evaluator while utilizing the text analysis capability of an AI model. [Means for solving the problem]

[0007] A skill evaluation device according to one embodiment of the present invention is a skill evaluation device that determines the skills of a subject by having an AI model evaluate a target text, and comprises an input unit that receives input of the target text from the subject, a control unit that controls the execution of inference processing by the AI ​​model, and an inference unit that executes the inference processing by the AI ​​model, wherein the evaluation process by the AI ​​model includes a first phase of measuring the quality of the description of the target text and a second phase of measuring the skills of the subject, and the control unit controls the device so as not to proceed to the second phase if, as a result of the measurement by the AI ​​model in the first phase, it is determined that the description of the target text does not meet a predetermined quality standard.

[0008] A skill evaluation method according to one embodiment of the present invention is a skill evaluation method for determining a person's skills by having an AI model evaluate a target text, comprising: an input step of receiving input of the target text from the person; a control step of controlling the execution of inference processing by the AI ​​model; and an inference step of executing the inference processing by the AI ​​model, wherein the evaluation process by the AI ​​model includes a first phase of measuring the quality of the description of the target text and a second phase of measuring the skills of the person, and in the control step, if the AI ​​model determines in the first phase that the description of the target text does not meet a predetermined quality standard as a result of the measurement, the method controls not to proceed to the second phase.

[0009] A program according to one embodiment of the present invention provides a computer that determines a person's skills by having an AI model evaluate target text, and the computer functions as an input unit that receives input of the target text from the person, a control unit that controls the execution of inference processing by the AI ​​model, and an inference unit that executes the inference processing by the AI ​​model, wherein the evaluation process by the AI ​​model includes a first phase that measures the quality of the description of the target text and a second phase that measures the skills of the person, and the control unit controls the system so as not to proceed to the second phase if, as a result of the measurement by the AI ​​model in the first phase, it is determined that the description of the target text does not meet a predetermined quality standard. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a skill evaluation device that can perform appropriate evaluations similar to those performed by human evaluators, while utilizing the text analysis capabilities of an AI model. [Brief explanation of the drawing]

[0011] [Figure 1] A figure showing an example of application of the skill evaluation device 1 according to an embodiment of the present invention. [Figure 2] A diagram showing the hardware configuration of the skill evaluation device 1 according to an embodiment of the present invention. [Figure 3] A diagram showing the configuration of a user terminal 2 according to an embodiment of the present invention. [Figure 4] A block diagram showing a functional module of a program executed by the processor 11 of a skill evaluation device 1 according to an embodiment of the present invention. [Figure 5] A sequence diagram of the first phase of the skill evaluation process by the skill evaluation device 1 according to an embodiment of the present invention. [Figure 6] A sequence diagram of the second phase of the skill evaluation process by the skill evaluation device 1 according to an embodiment of the present invention. [Figure 7] Sequence diagram of the overall judgment of the skill evaluation process by the skill evaluation device 1 according to an embodiment of the present invention. [Modes for carrying out the invention]

[0012] Figure 1 shows an example of the application of a skill evaluation device 1 according to an embodiment of the present invention. The skill evaluation device 1 is connected to a user terminal 2 via a communication network N. The communication network N may be any of the following: the internet, LAN, dedicated line, telephone line, corporate network, mobile communication network, Bluetooth®, WiFi (Wireless Fidelity), other communication lines, or a combination thereof, and may be wired or wireless.

[0013] Skill evaluation device 1 can be used, for example, by an employee of a company to diagnose their own professional skills. Employees can receive a skill assessment generated by AI by inputting information such as their work history via user terminal 2.

[0014] Skill evaluation device 1 may be installed in a company's information management center or similar location, or it may be built on the cloud. Skill evaluation device 1 may be a general-purpose computer, and may consist of a single computer or multiple computers distributed over a communication network N.

[0015] User terminal 2 is a terminal used by employees who wish to undergo a skills assessment using skill evaluation device 1, and can be a general-purpose device such as a smartphone, tablet, personal computer (PC), or notebook PC. When using skill evaluation device 1, it can be used either through a dedicated application or by accessing a dedicated website. Although only one user terminal 2 is shown in Figure 1, in reality, skill evaluation device 1 can be used from multiple user terminals 2.

[0016] Fig. 2 is a block diagram showing the hardware configuration of the skill evaluation apparatus 1. As shown in Fig. 2, the traffic volume prediction apparatus 1 includes a processor 11, a main memory 12, an input / output interface 13, a communication interface 14, and a storage device 15. The storage device 15 is a computer-readable recording medium such as a semiconductor memory (e.g., volatile memory or non-volatile memory) or a disk medium (e.g., magnetic recording medium or magneto-optical recording medium). The storage device 15 stores programs to be executed by the processor 11, various types of data, and the like. Various functions are implemented when the programs are read from the storage device 15 into the main memory 12 and interpreted and executed by the processor 11.

[0017] Fig. 3 is a block diagram showing the configuration of the user terminal 2. As shown in Fig. 3, the user terminal 2 includes a processor 21, an input device 22 such as a keyboard, a mouse, various operation buttons, and a touch panel, a display device 23 such as a liquid crystal display, a communication interface 24 for connecting to a communication network N, and a storage device 25 such as a disk drive or a semiconductor memory (ROM, RAM, etc.). The storage device 25 may store various programs executed by the processor 21 and various data.

[0018] Fig. 4 is a block diagram showing functional modules of a program executed by the processor 11 of the skill evaluation apparatus 1. As shown in Fig. 4, the functional modules executed by the processor 11 of the skill evaluation apparatus 1 include an input unit 101, a control unit 102, an inference unit 103, and an output unit 104. Note that part of the functions of the skill evaluation apparatus 1 may be implemented by the user terminal 2.

[0019] The input unit 101 provides an interface that accepts text data (target text) relating to work experience for performing skill evaluation from an employee or the like (target person) to be evaluated. In the present embodiment, the input unit 101 provides a structured input interface based on the STAR framework.

[0020] Specifically, the input unit 101 may be configured to display, on the screen of the user terminal 2, input fields corresponding to the four components of the STAR framework, namely Situation, Task, Action, and Result, and accept inputs. As an example of the input fields, the following three fields can be cited.

[0021] (Field 1) Background·Task (Situation / Task): The purpose of the activity, the situation you were in, and the issues to be solved. (Field 2) Action·Role (Action): What specifically did the evaluated person do, and what role did they play. (Field 3) Outcome·Result (Result): Quantitative and qualitative outcomes obtained through actions, as well as insights gained therefrom.

[0022] The control unit 102 has a function of managing the overall flow of the evaluation process. The control unit 102 has a separated structure in which the evaluation process is divided into a "quality verification phase" (first phase) for judging the quality of the target text, and a "skill evaluation phase" (second phase) for evaluating the skill of the target person. In the first phase, the control unit 102 implements gate passage judgment (logical interlock) that does not allow transition to the second phase unless it is determined that the quality of the target text satisfies a predetermined reference value.

[0023] The evaluation criteria in the first phase can be changed and set according to the skills and systems to be diagnosed. For example, optimal quality standards can be applied according to the purpose of evaluation: for expertise diagnosis, the condition is that "concreteness of discussion" and "consistency of logic" satisfy certain standards; for digital skill diagnosis, the condition is that "concreteness of discussion" and "satisfaction of question requirements" satisfy certain standards.

[0024] Furthermore, if the target text does not meet the quality standards, the control unit 102 will request the user to ask further questions and resubmit the target text up to a predetermined number of times (information completion loop). If the standards are still not met even after reaching the maximum number of attempts, the control unit 102 will determine the evaluation result (not met, grade, level, etc.) in light of the standards in the first phase, and then deterministically branch whether to interrupt the evaluation process or forcibly transition to the second phase. This structurally prevents problems such as incomplete target texts moving to the second phase without accompanying evaluation results, or conversely, the AI ​​performing excessive contextual completion (hallucination) and treating the text as meeting the standards.

[0025] Furthermore, in the second phase, the control unit 102 separates the AI-based inference processing (measurement) from the system-based overall judgment processing and performs the following two-stage system judgment processing on the measurement results by the AI ​​in order to achieve information concealment between the two.

[0026] (1) Individual skill assessment The system receives individual skill measurements (scores and level estimates) generated by the AI, compares them with internally managed reference values ​​(thresholds, etc.), and determines whether or not the skill has been acquired and its level. This process does not need to be performed for all skills at once; it can be performed on individual skills based on the user's choice.

[0027] (2) Overall Judgment Based on multiple previously confirmed and accumulated "individual skill assessment results," the AI ​​applies a concealed "scoring table for overall assessment" and "passing criteria logic" to perform the final overall assessment. The timing of this overall assessment can be set according to the purpose of use. For example, in the case of digital skill assessment, the overall assessment may be performed automatically each time an individual assessment is performed. In the case of expertise assessment, the overall assessment may be triggered by a user's arbitrary instruction.

[0028] By concealing the logic behind "which skills are important" and "how to achieve overall success" from the AI, the risk of the AI ​​distorting individual judgments (due to favoritism or the halo effect) by being conscious of the final result can be structurally eliminated.

[0029] The inference unit 103 has the function of analyzing input text data (target text) using a generating AI and calculating various scores and evaluation indicators for skill evaluation. The generating AI is composed of AI models such as LLM. When performing an evaluation, the inference unit 103 reads the definition information of the target skills stored in the storage device 15 and performs an analysis including whether the input work history data matches these definitions or whether the defined knowledge and skills are being demonstrated.

[0030] In this embodiment, the inference unit 103 only performs "measurement" of each evaluation indicator and does not perform "overall judgment". Specifically, for each skill, it only makes a judgment based on the evaluation scale (achievement / failure, level value, rank, etc.) set by the manager or other person. On the other hand, no information is provided as to how these measurement results are ultimately aggregated and an overall judgment is output.

[0031] Furthermore, the inference unit 103 performs statistical aggregation processing when measuring evaluation metrics. In the gate passage determination by logical interlock by the control unit 102, and in the measurement processing in the first and second phases under the separated structure, N (for example, 3) parallel inferences are performed, and the results are statistically aggregated. Specifically, aggregation may be performed by majority voting or by adopting the median. By performing statistical aggregation processing, not only is the accuracy of the measurement results improved, but it can also function as a multi-layered defense to eliminate the risk of "unfair gate breaches" due to accidental hallucination (generation not based on facts) by the AI, and the risk of "logically inconsistent descriptions being injected into the system logic."

[0032] The output unit 104 is an interface that presents information such as evaluation results to the person being evaluated and the manager. (Presentation of revised questions) If the control unit 102 determines that the quality of the target text does not meet the standards, the output unit 104 displays additional questions along with the reason (pointing out the shortcomings) to encourage the person being evaluated to improve the quality of the input data through corrections. This ensures that the amount of information necessary for skill evaluation in the second phase is secured.

[0033] (Display of results and reasons for judgment) The output unit 104 displays the final skill level assessment result and pass / fail result, as well as the assessment reason (feedback text) generated by the inference unit 103.

[0034] The storage device 15 is equipped with a database or storage area that holds definition data and historical data necessary for the evaluation process. Specifically, it stores the following information:

[0035] (Skill definition information) This is master data that defines the details of each skill that will be evaluated.

[0036] (Evaluation criteria definition information) This system manages the definitions of each indicator used for evaluation. In addition to standard indicators (satisfaction of question requirements, specificity of arguments, validity of content, logical consistency), it includes evaluation axes (rubrics, competency models, etc.) that can be independently defined and set by the user company.

[0037] (Evaluation sequence information (configuration information)) This setting defines the number of phases that make up the evaluation process, as well as the conditions for passing each phase (specificity of the argument, logical consistency, etc.) and the pass threshold. This setting allows the criteria for quality gates to be varied according to the diagnostic purpose.

[0038] (Judgment logic information (scoring table)) This information is hidden from the AI ​​and holds the scoring logic and weighting information used for the overall evaluation.

[0039] (Inference log data) Data that records the processing details at each stage of the evaluation process in chronological order.

[0040] Next, the flow of skill evaluation by the skill evaluation device 1 will be explained using the sequence diagrams in Figures 5 to 7. The numbers in parentheses indicate the processing steps. (Phase 1) First, the processing of the first phase will be explained using Figure 5. The person being evaluated operates the user terminal 2 to select the skill to be evaluated and input the target text (STAR ​​framework) (Step 1). Next, the input unit 101 structures the input text and supplies it to the control unit 102 (Step 2), and the first phase begins.

[0041] The control unit 102 instructs the inference unit 103 to measure the quality of the target text (specificity, completeness, etc.) (Step 3). The inference unit 103 performs N inferences regarding the measurement of the quality of the target text and statistically aggregates the results of the N inferences (Step 4). The inference unit 103 supplies the aggregated measurement results to the control unit 102 (Step 5).

[0042] The control unit 102 performs a gate passage determination based on the measurement results received from the inference unit 103 (step 6). If the determination shows that the target text meets a predetermined quality standard, the system proceeds to the next phase (second phase). On the other hand, if the standard is not met, the system presents the user with a new question up to a predetermined number of times (step 7). For example, a question such as "The explanation of XX is insufficient. Please describe the content in detail." is displayed on the user terminal 2. The user inputs an additional answer to the question (step 8). The control unit 102 executes steps 3 to 6, and if the corrected text meets the standard, the system proceeds to the next phase; otherwise, it presents the new question again. If the target text still does not meet the standard even after reaching the maximum number of re-questions, the control unit 102 refers to the branch setting stored in the storage device 15 (step 9). If the branch setting is "end of process," an error message is output to the user terminal 2 and the process is terminated (step 10). If the branch setting is "forced transition," the system forcibly proceeds to the next phase.

[0043] (Phase 2) Next, the processing of the second phase will be explained using Figure 6. The control unit 102 instructs the estimation unit 103 to measure the skills to be evaluated selected by the subject (step 11). Based on the instruction, the inference unit 103 performs measurement of each evaluation indicator (step 12). The inference unit 103 performs N inferences, statistically aggregates the results of the N inferences, and supplies the measurement results (score) to the control unit 102 (step 13). The control unit 102 compares the measured values ​​(scores and level estimates) of each skill output by the AI ​​with reference values ​​(thresholds, etc.) stored in the memory device 15 to determine whether each skill has been acquired (pass or fail) and to determine the level (steps 14, 15). The control unit 102 saves the judgment results of the individual skills in the memory device 15 (step 16).

[0044] (Overall assessment) Next, the process of the overall judgment will be explained using Figure 7. The control unit 102 refers to the execution timing of the overall judgment stored in the storage device 15 (step 17) and performs the overall judgment according to the set timing. The overall judgment is performed by applying the "scoring table for overall judgment" and "passing criteria logic," which are hidden from the AI, based on the "individual skill judgment results" stored in the storage device 15 (steps 18-20).

[0045] The control unit 102 outputs the "individual skill assessment result" and the "overall assessment result" to the user terminal 2. Alternatively, depending on the settings, it may display only the individual skill assessment result (steps 21, 22). If the overall assessment is set to be performed when the subject gives instructions, when the subject gives an instruction to perform the overall assessment via the user terminal 2 (step 23), the input unit 101 notifies the control unit 102 of the execution request (step 24). The control unit 102 retrieves the previously accumulated individual skill assessment results from the storage device 15 (step 25), applies the overall assessment logic, and performs the overall assessment (step 26). The control unit 102 outputs the overall assessment result, and the result is displayed on the user terminal 2 (step 27).

[0046] As described above, according to this embodiment, the skill evaluation process is divided into a first phase, which measures the quality of the description of the target text, and a second phase, which measures the skills of the subject. In the first phase, the process is controlled so that it does not proceed to the second phase until it is determined that the description of the target text meets a predetermined quality standard. This prevents the generating AI from making an incorrect skill evaluation based on text that does not contain sufficient content to accurately evaluate skills.

[0047] Furthermore, since the generation AI estimation is performed multiple times and the measurement results are statistically aggregated, outliers from the AI ​​measurement can be eliminated, improving accuracy.

[0048] Furthermore, in the second phase, which involves evaluating the skills of the participants, the generating AI was only allowed to measure individual skills, and the criteria for overall evaluation (such as scoring logic and weighting) were not provided to the generating AI. This eliminates the "halo effect," which is an error in the generating AI that leads to overly lenient evaluation of details based on overall impressions. This allows for a more accurate and fair evaluation than if the AI ​​were given all the judgment criteria and made to make an overall judgment.

[0049] Furthermore, in the first phase, if the quality of the target text is determined not to meet the standards, a limit is placed on the number of times the question can be reassigned, and this is controlled by the system. This ensures the quality of the target text while preventing a situation where the AI ​​endlessly repeats the question.

[0050] The following describes examples of applications of the present invention. (Case Study 1: Employee Digital Skill Assessment) Case Study 1 is a diagnostic model for visualizing the extent to which employees are demonstrating their digital skills, such as security and data science. 1) Prerequisites (Target audience and questions) • Target skills The person being evaluated selects a target skill corresponding to their area of ​​expertise from a skill set such as "intelligence" or "data analysis." The system retrieves a detailed list of skills corresponding to the selected area (e.g., "mitigating cognitive bias," "ensuring the integrity of evidence," etc.) and information on "evaluation items," "required knowledge," and "required skills" associated with each skill from storage device 15. Initial Questions Questions are presented regarding specific activities in which the target skills were demonstrated, asking about the "purpose of implementation," "background of the issue," "target and scope of implementation," "structure of the activity," "your role," and "content of implementation," as well as the "output (results)" related to the "evaluation items" obtained from memory device 15. 2) Characteristics of the settings • Evaluation criteria In the first phase, the evaluation is based on "satisfaction of the question requirements" and "specificity of the argument" to determine whether the written content contains enough information to warrant evaluation. In the second phase, "validity of content" and "logical consistency" are applied to evaluate whether the described actions and knowledge are appropriate in light of the skill requirements (evaluation items, knowledge, and skills) defined in memory device 15. • Transition control To encourage skill improvement among participants, the system is set to automatically transition to Phase 2 even if the criteria are not met in Phase 1 (reaching the upper limit of the re-question loop). • Processing flow The inference unit 103 first evaluates the formal completeness and specificity of the input text. If there are deficiencies, it asks further questions, but even if the specificity remains low, it proceeds to the second phase. Here, for example, regarding the skill "mitigating cognitive bias," it analyzes whether the user's description appropriately takes into account "knowledge of intelligence recruitment requirements." The control unit 102 integrates all of these evaluation results and, in the overall judgment, outputs not only pass or fail but also detailed feedback such as "there is an error in the application of knowledge" or "there is a lack of specificity."

[0051] (Case Study 2: Expertise Assessment for Professional Certification) Case Study 2 is a diagnostic model for certifying the skill level of specific technical roles, such as IT service managers. • Prerequisites (Target audience and questions) • Target skills The person being evaluated selects the job title / specialty field that applies to them from among several options (e.g., Job Title: IT Service Management, Specialty Field: System Administration). The system retrieves a detailed list of skills corresponding to the selected job title / specialty field, along with questions associated with each skill, from storage device 15. Initial Questions As a prerequisite, applicants are required to input "project information," "job information (job summary, position / role, etc.)," ​​etc. Then, for each individual skill that constitutes the job (e.g., troubleshooting, information asset management, problem management), applicants are asked to select their own "self-assessment level (e.g., level 4)" and describe the specific "skill performance" that supports that assessment. • Features of the settings • Evaluation criteria In the first phase, the "logical consistency" and "specificity of the argument" of the entire description are evaluated. In the second phase, the level assessment criteria (rubric) defined for each skill item are compared with the "self-assessed level" selected by the subject and the described "skill performance" to individually and absolutely evaluate whether the subject's ability has reached the self-assessed level. • Transition control Even if the criteria are not met in Phase 1, the system will forcibly transition to Phase 2, where evaluations will be performed on all skill items. • Processing flow The inference unit 103 reads case information, job information, etc., as context and evaluates the "specificity of the argument" of the description. For example, if the description is abstract and the results are unclear, the control unit 102 generates a revised question such as, "In problem management, please describe the specific methods used to identify the root cause and the quantitative results obtained as a result," to encourage the description to become more specific. This process of generating revised questions and evaluating additional answers is repeated up to two times. After that, a forced transition is performed to determine the level of all skill items. Finally, the control unit 102 integrates all of these evaluation results to make a level determination. Specifically, it compares the "self-assessed level" declared by the subject with the "actual level" determined by the AI ​​and determines the level (e.g., self-assessed level 4, but the description is judged to be equivalent to level 3). It also outputs a diagnostic report that includes the basis for the level determination.

[0052] As described above, the present invention can be widely used as a personnel evaluation system, skill certification system, automated scoring system for qualification examinations in companies, government agencies, educational institutions, etc., as well as a skill assessment system for skill matching platforms for freelancers, etc.

[0053] It should be noted that the present invention is not limited to the embodiments described above, and can be implemented in various other forms without departing from the spirit of the invention. For this reason, the above embodiments are merely illustrative in all respects and should not be interpreted restrictively. For example, the order of each processing step described above can be arbitrarily changed or executed in parallel, as long as there is no inconsistency in the processing content. [Explanation of symbols]

[0054] 1...Skill evaluation device, 2...User terminal, 11...Processor, 12...Main memory, 13...Input / output interface, 14...Communication interface, 15...Storage device, 21...Processor, 22...Input device, 23...Display device, 24...Communication interface, 25...Storage device, 101...Input unit, 102...Control unit, 103...Inference unit, 104...Output unit

Claims

1. A skill evaluation device that determines the skills of a subject by having an AI model evaluate the target text, An input unit that receives input of the target text from the aforementioned subject, A control unit that controls the execution of processing by the AI ​​model, The system comprises an inference unit that analyzes the target text and performs processing using the AI ​​model, The processing by the AI ​​model includes a first phase of calculating measurement results regarding the quality of the description of the target text, and a second phase of calculating measurement results regarding the skills of the subject. The control unit, A skill evaluation device that, in the first phase, determines, based on the measurement results regarding the quality of the description of the target text, that the description of the target text does not meet predetermined quality standards, and controls the device to prevent it from proceeding to the second phase.

2. The control unit, In the first phase, if the description of the target text does not meet the quality standards, the skill evaluation device according to claim 1 accepts the resubmission of the corrected target text from the subject up to a predetermined number of times, and causes the inference unit to perform a measurement of the quality of the description of the target text again, and if the quality standards are not met even after the predetermined number of times, the skill evaluation is terminated or the device is forced to move to the second phase.

3. The inference unit, In the second phase described above, the measurement results for one or more evaluation items are output, The control unit, The skill evaluation device according to claim 1, wherein the inference unit performs an overall judgment based on the measurement results for each of the one or more evaluation items, and presents the result to the subject.

4. The inference unit, The skill evaluation device according to claim 1, wherein in the first phase, measurements regarding the quality of the target text are performed multiple times, and the results of each measurement are statistically aggregated to obtain the measurement results.

5. The inference unit, The skill evaluation device according to claim 1, wherein in the second phase, the skills of the subject are measured multiple times, and the results of each measurement are statistically aggregated to obtain the measurement results.

6. A computer-based skill evaluation method that determines a person's skills by having an AI model evaluate the target text, The computer performs an input step in which it receives input of the target text from the target person, The computer performs a control step that controls the execution of processing by the AI ​​model, The computer comprises an inference step of analyzing the target text and executing processing by the AI ​​model, The processing by the AI ​​model includes a first phase of calculating measurement results regarding the quality of the description of the target text, and a second phase of calculating measurement results regarding the skills of the subject. In the control process described above, A skill evaluation method that, in the first phase, controls the process to not proceed to the second phase if it is determined, based on the measurement results regarding the quality of the description of the target text, that the description of the target text does not meet predetermined quality standards.

7. By having an AI model evaluate the target text, a computer can determine the skills of the subject. An input unit that receives input of the target text from the aforementioned subject, A control unit that controls the execution of processing by the AI ​​model, This unit functions as an inference unit that analyzes the target text and executes processing by the AI ​​model. The processing by the AI ​​model includes a first phase of calculating measurement results regarding the quality of the description of the target text, and a second phase of calculating measurement results regarding the skills of the subject. The control unit, A program that, in the first phase, if it is determined that the description of the target text does not meet a predetermined quality standard based on the measurement results regarding the quality of the description of the target text, controls the program to prevent it from proceeding to the second phase.

Images