Information processing method and program for evaluating the personal characteristics of a person being evaluated

Abstract

To efficiently obtain an appropriate evaluation for each person being evaluated, by changing the information presented next according to the evaluator's response or the progress of the evaluation. [Solution] A computer operating in conjunction with an AI model selects evaluation items from among multiple evaluation items to evaluate the person being evaluated based on a first criterion, generates and outputs information to be presented to the person being evaluated based on a second criterion corresponding to the selected evaluation items, obtains response information, applies the evaluation criterion corresponding to the evaluation item to the response information to output an evaluation value of the person's characteristics, updates the degree to which the information necessary for evaluation has been obtained based on the response information, and changes the information to be presented next based on the updated degree. The AI ​​model is equipped with storage for at least one of the following: evaluation items, information to be presented, evaluation criteria, the first criterion, the second criterion, or information indicating response information.

Description

Technical Field

[0001] The present invention relates to an information processing method and a program for evaluating the personal characteristics of an evaluated person.

Background Art

[0002] Personnel assessment is known as a method for objectively evaluating the personal characteristics of an evaluated person in order to contribute to decision-making such as recruitment, placement, promotion or upgrading, education and training, or self-improvement. Personnel assessment is used, for example, in the selection of candidates for management positions, and the potential characteristics that an evaluated person can exhibit in future duties can be evaluated.

[0003] Conventionally, in personnel assessment, a plurality of exercises such as an in-basket exercise, a policy-making exercise or an analysis presentation exercise, an interview exercise, a group discussion exercise, or an essay or description exercise are used, and a trained human assessor observes the actions or work products of the evaluated person and evaluates them according to evaluation criteria. In one example, an evaluated person is given an exercise task that simulates a practical situation, and the actions of the evaluated person who undertakes the task are observed.

[0004] In such a method, the concept of predicting future behavior based on observed behavior, the concept that the essential characteristics of an evaluated person are likely to be expressed in a situation where a high load, time constraints, or an unfamiliar role is given, are known as the theoretical background of the evaluation. In one example, the concept that past behavior is an indicator for predicting future behavior, and the concept that characteristics such as the thinking style, values, or tolerance to load of an evaluated person are likely to be expressed in a situation with a strong load can be referred to as the theoretical background of the evaluation.

[0005] In recent years, methods have been proposed that use artificial intelligence (AI) to score or evaluate audio recordings of interviewees' responses or utterances (Patent Document 1, Non-Patent Document 1). Methods have also been proposed that select the next question to be presented based on the scoring results of the interviewee's previous responses (Patent Document 1). On the other hand, methods are known that extract the interviewee's facial expressions, posture, or gaze from video footage and present scenes that human assessors should focus on, thereby supporting observation by human assessors (Non-Patent Document 2). [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] U.S. Patent Application Publication No. 2015 / 0269529 [Non-patent literature]

[0007] [Non-Patent Document 1] HireVue, Assessment Software, https: / / www.hirevue.com / platform / assessment-software (accessed June 3, 2020) [Non-Patent Document 2] Riku Arakawa, Hiroki Yakura, Artificial Intelligence for Human Assessment: What Do Professional Assessors Need? https: / / arxiv.org / abs / 2204.08471 (Accessed June 3, 2026) [Overview of the project] [Problems that the invention aims to solve]

[0008] The disclosure technology aims to efficiently obtain appropriate evaluations for each person being evaluated. [Means for solving the problem]

[0009] The disclosed technology is an information processing method in which a computer operating in conjunction with an AI model outputs evaluation values ​​for the characteristics of a person being evaluated, comprising: selecting evaluation items from a plurality of evaluation items to evaluate the person being evaluated based on a first criterion; generating and outputting information to be presented to the person being evaluated based on a second criterion corresponding to the selected evaluation items; acquiring response information to the information presented to the person being evaluated; applying evaluation criteria corresponding to the evaluation items to the response information to output evaluation values ​​for the person being evaluated; updating the degree to which information necessary for evaluation has been acquired for the evaluation items based on the response information; and changing the information to be presented to the person being evaluated next based on the updated degree, wherein the AI ​​model is equipped with storage for at least one of the evaluation items, the information to be presented to the person being evaluated, the evaluation criteria, the first criterion, the second criterion, or information indicating the response information. [Effects of the Invention]

[0010] According to this embodiment, by changing the information presented to the person being evaluated based on the extent to which the necessary information for evaluation has been obtained for each evaluation item, it is possible to efficiently obtain the necessary information for evaluation and obtain a more appropriate evaluation for each person being evaluated within limited resources or time. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a diagram showing an example of the overall processing flow of the information processing method according to the embodiment. [Figure 2] Figure 2 shows an example of a processing flow that applies evaluation criteria to response information to output evaluation values ​​for the person being evaluated. [Figure 3] Figure 3 shows an example of a processing flow for generating and outputting confirmation items, etc., for evaluation items where the extent to which the necessary information for evaluation has been acquired does not meet the predetermined conditions. [Figure 4]Figure 4 shows an example of a processing flow for generating and outputting information that makes the claims and other statements included in the response information more specific. [Figure 5] Figure 5 shows an example of the hardware configuration of a computer according to this embodiment. [Figure 6] Figure 6 shows an example of a functional configuration according to the embodiment. [Modes for carrying out the invention]

[0012] Embodiments will be described with reference to the drawings. The following description is illustrative, and the disclosed technology is not limited to the embodiments described below. The configurations described in each embodiment may be combined with each other to the extent that they do not conflict technically. Furthermore, each step (each process) disclosed herein may be performed in any order, or some may be performed in parallel, as long as it does not conflict technically. However, this does not apply if the order or parallelism of the steps has technical significance.

[0013] The information processing method according to this embodiment is executed by a computer operating in conjunction with an AI model. The computer outputs evaluation values ​​for the personality traits of the person being evaluated using the hardware resources shown in Figure 5, which will be described later. In one example, the computer may be implemented as a server, a personal computer, a tablet terminal, or a combination thereof, and may be connected to a terminal used by the person being evaluated via a communication network. In one example, the evaluation may be conducted in a form where the person being evaluated does not gather in one place, by connecting to a terminal used by the person being evaluated via a communication network.

[0014] An "evaluated person" refers to a person whose personal characteristics are being evaluated. An evaluated person may include, but is not limited to, training participants, students, examinees, candidates, employees, management candidates, or employment candidates. For example, an evaluated person may be a management candidate, and the computer may execute an information processing method according to the embodiment for the selection of management candidates.

[0015] Person characteristics refer to a concept that includes at least one of the abilities, skills, knowledge, aptitudes, competencies, behavioral characteristics, behavioral tendencies, thinking characteristics, interpersonal characteristics, job performance characteristics, or job suitability of the evaluated person, and are not limited thereto. In one example, person characteristics may include the tendency of the evaluated person's actions when dealing with tasks, the tendency of how to interact with others, or the tendency of how to think. In one example, person characteristics can be sorted into at least one of the categories related to the business skills of the evaluated person, the categories related to abilities or qualifications, and the categories related to thinking or behavioral characteristics. Each category may include a plurality of specific evaluation items at a lower level of the category. These categories and evaluation items are all examples included in person characteristics and do not limit person characteristics to the above-mentioned categories or evaluation items.

[0016] An evaluation item is an indicator that shows the person characteristics of the evaluated person. Evaluation items may be classified into evaluation axes (also referred to as major classifications), which are higher-level categories that group a plurality of evaluation items. In one example, each evaluation axis includes a plurality of evaluation items. That is, evaluation axis and evaluation item are terms in a hierarchical relationship and are not synonymous. In one example, an evaluation item may be stored in the storage unit as a data structure that associates identification information, name, information indicating the evaluation axis to which it belongs, information indicating the definition, and information indicating the corresponding evaluation criteria.

[0017] Evaluation criteria refer to information that shows at least one of the definition, concept, keyword, element concept, priority (weight) of element concepts, score, or action example corresponding to the score for each evaluation item. Evaluation criteria can be referred to when outputting an evaluation value from response information. In one example, evaluation criteria may be stored in the storage unit as a tabular data structure that associates a plurality of element concepts, the corresponding priority (weight) for each element concept, a plurality of scores, and action examples corresponding to each score for each evaluation item. Evaluation criteria do not have to include all of the above elements and may include some of them according to the operation.

[0018] The information presented to the evaluated person (hereinafter also referred to as presented information) is a concept that includes at least one of questions, tasks, cases, instructions, options, virtual work scenarios, statements of the counterpart, counterarguments, additional conditions, dilemma situations, emails, chats, reports, consultation contents, documents, images, voices, or videos, and is not limited thereto. In one example, the presented information is generated as data including a text, a list of options, an image or video file, or an audio file, and may be output via a display unit, an audio output unit, a video output unit, or the like.

[0019] The presented information may be output with an expression mode such as the tendency or intensity of its expression. In one example, the tendency or intensity of expression may be reflected in the text, voice, expression, posture, gaze, or response interval of the presented information. As will be described later, the expression mode is a value (label) indicating the expression conditions set by the computer for the presented information, and does not indicate the internal state of the computer or the AI model.

[0020] The response information is a concept that may include character information input by the evaluated person, voice information indicating the speech of the evaluated person, character information generated from the speech, information indicating the expression, posture, or gaze of the evaluated person, or information indicating the response time to the information presented to the evaluated person, and is not limited thereto. The response information may include non-verbal information such as nodding or making a sound. In one example, the response information may be recorded in the storage unit as a data structure associating the identification information of the evaluated person, the identification information of the presented information, the information indicating the content of the response, and the information indicating the acquisition time of the response.

[0021] Nonverbal information that may be included in response information may include, for example, visual elements, auditory elements, or bodily sensation elements. Visual elements may include, for example, information indicating the facial expressions of the person being evaluated. Auditory elements may include, for example, information indicating the pitch, volume, quality, speed, intonation, pauses, or responses of the person being evaluated. Bodily sensation elements may include, for example, information indicating the gestures, body language, or posture of the person being evaluated. This information represents features extracted by the computer from captured video or acquired audio, and does not directly indicate the inner state of the person being evaluated.

[0022] The first criterion is the standard that the computer refers to in order to select the evaluation items from among several evaluation items to evaluate the person being evaluated. The first criterion may, for example, include conditions indicating the extent to which multiple evaluation items are covered, the amount of interaction remaining with the person being evaluated, or a combination of these. The specific content of the first criterion will be described later. In this specification, interaction with the person being evaluated may be referred to as interaction with the person being evaluated or engagement with the person being evaluated, and these refer to the same thing. Interaction with the person being evaluated refers to mutual interaction that includes presenting information to the person being evaluated and obtaining the person's response to that information. The amount of interaction with the person being evaluated refers to the amount that can be spent on interaction with the person being evaluated, and may be referred to as the amount of interaction with the person being evaluated or the amount of engagement with the person being evaluated. The amount of interaction with the person being evaluated may, for example, be expressed as the total amount of predetermined time for interaction with the person being evaluated, or the total amount of predetermined number of interactions with the person being evaluated. The number of times may, for example, be the number of questions or tasks that can be presented to the person being evaluated, or the number of exchanges between the presentation of information and the response. The amount remaining for interaction with the person being evaluated refers to the portion of the total amount of interaction with the person being evaluated that has not yet been spent, and may be referred to as the amount remaining for interaction with the person being evaluated or the amount remaining for engagement with the person being evaluated. The amount remaining for interaction with the person being evaluated may, for example, be expressed as the remaining time out of a predetermined amount of time, or the remaining number out of a predetermined number of times. In this specification and in the claims, the term "interaction with the person being evaluated" is used as the standard, and "interaction with the person being evaluated" and "engagement with the person being evaluated" are synonymous paraphrases.

[0023] The second criterion is the standard that the computer refers to in order to generate the information presented to the person being evaluated. The second criterion may include, for example, evaluation values ​​based on already acquired response information, the extent to which the necessary information for evaluation has been acquired, or conditions indicating the reliability of the evaluation values. The specific details of the second criterion will be described later.

[0024] The degree to which information necessary for evaluation has been acquired (hereinafter also simply referred to as "degree") is an indicator that shows to what extent sufficient response information has been acquired for each evaluation item to output an evaluation value. The degree can be expressed in any form, such as a value, indicator, state, or a combination thereof, and is not limited to a specific expression format. For example, the degree may be stored in the memory as a value or state determined according to the amount of response information acquired, the number of evaluation grounds, or the confidence level of the evaluation value for each evaluation item. The updating of the degree will be described later.

[0025] An evaluation value refers to information that shows the results of an evaluation of a person's characteristics, output by a computer. Evaluation values ​​can be expressed in any format, such as points, grades, labels, or a combination thereof. An evaluation score is a point or value corresponding to a rating, output by a computer as a result of applying evaluation criteria to response information for an evaluation item, and is one form of an evaluation value. The reliability of an evaluation value is an indicator of the certainty of the output evaluation value, and can be expressed as a value determined according to the amount of response information used to output the evaluation value, the number of evaluation criteria, or the consistency of the response information, for example.

[0026] The term "AI model" is a concept that may include, but is not limited to, artificial intelligence, machine learning models, trained models, neural networks, deep learning models, generative AI, or large-scale language models. For example, an AI model may be configured to output text, evaluation values, or confirmation requests based on input information.

[0027] The memory imposed on an AI model is a concept that may include, but is not limited to, training data, teacher data, parameters of the trained model, prompts, information indicating procedures, information indicating evaluation procedures, information indicating output formats, reference information, or information indicating validation rules. Specific examples of information imposed on an AI model will be discussed later.

[0028] Additional information refers to information that shows the relationship between response information and evaluation items, and is information that a computer can output when outputting evaluation values. Specific examples of additional information will be discussed later.

[0029] The assessment according to this embodiment is not limited to a single interview, but may be conducted within an assessment unit that includes multiple exercises. For example, the assessment may include a hierarchy of training, exercises, tasks, responses, and evaluation. For example, one assessment unit may include multiple exercises such as an in-basket exercise and a group discussion exercise, in which multiple tasks are presented to the person being evaluated in each exercise, and responses to each task are obtained and evaluated. For example, the computer may select and conduct some of the exercises from among the multiple exercises according to the operation.

[0030] Figure 1 is a diagram showing an example of the overall processing flow of the information processing method according to the embodiment. Below, the processes executed by the computer will be described according to steps S101 to S106. Each process can be realized using hardware resources such as the CPU and memory unit shown in Figure 5, which will be described later, and the functional configuration shown in Figure 6, which will be described later. Below, for the purpose of explanation, an example of a series of processes will be described using evaluation items such as "cooperation" belonging to the evaluation axis "interpersonal" and "creativity" belonging to the evaluation axis "thinking" as examples.

[0031] In step S101, the computer selects evaluation items from among several evaluation items to evaluate the person being evaluated, based on a first criterion. For example, the computer reads multiple evaluation items and the first criterion from the memory unit, selects an evaluation item, and records the result of the selection in the memory unit. For example, at the start of the evaluation, since there are many evaluation items that have not yet been selected, the computer may select the evaluation item "cooperation" which belongs to the evaluation axis "interpersonal" based on the first criterion.

[0032] In step S102, the computer generates and outputs information to present to the person being evaluated, based on a second criterion, corresponding to the selected evaluation item. For example, the computer inputs information indicating the evaluation item and the second criterion into the AI ​​processing unit, retrieves the presentation information output from the AI ​​processing unit, and outputs it from the display unit or audio output unit, etc. For example, in response to the evaluation item "collaboration," the computer may output a hypothetical work scenario in which a problem is solved in cooperation with stakeholders, along with the statements of the other party, as presentation information. Specifically, the instruction, "You need to meet the deadline while coordinating with the relevant departments. How will you proceed?" and the statements of the other party may be output as presentation information.

[0033] In step S103, the computer acquires response information to the information presented to the person being evaluated. In one example, the computer acquires the response information via an input unit, an audio processing unit, or an image processing unit, etc., and records it in a storage unit. For example, the computer may acquire as response information the character information entered by the person being evaluated, audio information representing the person being evaluated's speech, character information generated from the speech, and information indicating the response time up to the response.

[0034] In step S104, the computer outputs an evaluation value for the person being evaluated by applying evaluation criteria corresponding to the evaluation items to the response information. In one example, the computer or AI processing unit applies evaluation criteria to the response information to output an evaluation value and records it in the memory unit. For example, the computer may associate an example of behavior included in the evaluation criteria for the evaluation item "cooperation" with information indicating the person being evaluated extracted from the response information, and output an evaluation value corresponding to the score. Specifically, if the person being evaluated mentions prior information sharing with relevant departments, the computer may associate it with an example of behavior with a high score corresponding to the element concept "communication," and output an evaluation value for the evaluation item "cooperation."

[0035] In step S105, the computer updates the degree to which the information necessary for evaluation has been acquired for each evaluation item, based on the response information. For example, the computer updates the degree based on the response information and records it in the memory. For instance, the computer may update the degree for the evaluation item "cooperation" to a high level because sufficient response information has been acquired, while maintaining the degree for the evaluation item "creativity" at a low level because there is insufficient response information.

[0036] In step S106, the computer changes the information it presents to the person being evaluated based on the degree of update. For example, the computer or AI processing unit changes the information presented next based on the degree of update. For instance, the computer may select a task corresponding to the evaluation item "creativity," which has a relatively low level, as the information to be presented next. These processes may be repeated as needed, and the evaluation may proceed. For example, the processes from step S101 to step S106 may be repeated until there is no more interaction remaining with the person being evaluated, or until the levels of multiple evaluation items meet predetermined conditions.

[0037] The processing in each of the above steps is a concrete implementation of software-based information processing using hardware resources. For example, each of the above steps can be realized by the CPU 1201 in Figure 5 reading and executing a program stored in the memory unit 1204, in cooperation with hardware resources such as the RAM 1203 in Figure 5, the input unit 1216, the display unit 1217 or the communication interface 1207 in Figure 5, or the AI ​​processing unit in Figure 6.

[0038] According to the embodiment, by repeating a series of processes while changing the information presented next in accordance with the evaluator's response or the progress of the evaluation, it is possible to efficiently acquire the information necessary for evaluation for multiple evaluation items under the constraints of a limited time or number of attempts. Furthermore, by proceeding with the evaluation while selecting the evaluation items and information to present to the evaluator in accordance with the evaluator's response or the progress of the evaluation, it is possible to conduct a deeper evaluation tailored to the level of proficiency or situation of each evaluator, even under the constraints of limited resources or time, compared to a uniform evaluation that uniformly presents predetermined questions or tasks, and to obtain a more appropriate evaluation for each evaluator.

[0039] Next, we will describe the selection of evaluation items based on the first criterion. For example, the first criterion may be a criterion for selecting evaluation items so as to cover multiple evaluation items. In this case, the computer may acquire multiple evaluation items, identify evaluation items that have already been selected for the person being evaluated, select evaluation items that have not yet been covered, and supply the results of the selection to generate the presentation information.

[0040] In one example, the computer may store information in its memory indicating whether each of several evaluation items has already been selected, and then select the next evaluation item to evaluate from among the evaluation items that have not yet been selected. For example, if evaluation items "Collaboration" and "Analysis" have already been selected, and evaluation items "Creativity" and "Control" have not yet been selected, the computer may select the unselected evaluation item "Creativity". In another example, the computer may aggregate the number of evaluation items that have already been selected for each evaluation axis, and then prioritize selecting evaluation items belonging to evaluation axes with a relatively small number of selected evaluation items.

[0041] According to the embodiment, the first criterion is a criterion for selecting evaluation items so as to cover multiple evaluation items, thereby reducing information bias towards specific evaluation items and making it easier to obtain the information necessary for evaluation across multiple evaluation items.

[0042] In another example, the criteria for selecting evaluation items may be determined based on the amount of interaction remaining with the person being evaluated. Here, the remaining amount may, in one example, be the amount of time remaining out of a predetermined time for interaction with the person being evaluated, or the number of interactions remaining out of a predetermined number of interactions with the person being evaluated. The predetermined number may, in one example, be the number of questions or tasks that can be presented to the person being evaluated. In another example, the predetermined number may be the number of round trips between the presentation of information and the response of the person being evaluated. The computer may determine the next evaluation items to be evaluated based on evaluation items not covered by the evaluation items already selected for the person being evaluated, and the remaining amount.

[0043] For example, the computer may determine whether the remaining amount is less than a predetermined value. If the remaining amount is less than the predetermined value, it may define an evaluation item that allows observation of multiple evaluation items with a single piece of information. If the remaining amount is greater than or equal to the predetermined value, it may define one evaluation item that has not yet been covered. The specific value or standard for the remaining amount may be determined as appropriate depending on the operation.

[0044] According to this embodiment, the criteria for selecting evaluation items are determined based on the amount of interaction remaining with the person being evaluated. This makes it easier to select evaluation items according to the time or number of times remaining for evaluation, thereby increasing comprehensiveness under limited constraints.

[0045] According to the embodiment, the remaining amount is the remaining time out of a predetermined time for interaction with the person being evaluated, or the remaining number of interactions out of a predetermined number of interactions. This makes it easier to select evaluation items based on the remaining time or number of interactions that can be used for evaluation, and to proceed with the evaluation under the constraints of a limited time or number of interactions.

[0046] Next, we will explain the second criterion. In one example, the second criterion may be determined based on an evaluation value derived from response information already obtained from the person being evaluated. In another example, the second criterion may be determined based on the extent to which the information necessary for evaluation has been obtained for the evaluation item, or on the reliability of the evaluation value. The computer may determine the second criterion based on at least one of these.

[0047] For example, if the evaluation value already obtained for a certain evaluation item is high and the confidence level of that evaluation value is high, the computer may not need to generate presentation information to evaluate that evaluation item in more detail. However, if the confidence level of the evaluation value is low, the computer may establish a second criterion to generate presentation information that makes it easier to obtain additional response information for that evaluation item. For example, if the confidence level of the evaluation value for the evaluation item "analysis" is low, the computer may establish a second criterion to generate a task that prompts the user to analyze more information.

[0048] According to the embodiment, the second criterion is determined based on evaluation values ​​derived from response information already obtained from the person being evaluated, making it easier to generate presentation information that corresponds to the evaluation status already obtained.

[0049] According to the embodiment, the second criterion is determined based on the extent to which the information necessary for evaluation has been obtained, or on the reliability of the evaluation value. This makes it easier to generate presentation information that facilitates the acquisition of additional response information for evaluation items where information is insufficient or the reliability of the evaluation value is low.

[0050] Next, the generation of presentation information will be explained. In one example, the computer may acquire evaluation items and a second criterion, select questions, select work scenarios, select documents, and output these as presentation information. The information presented to the person being evaluated may include at least one of the following: questions, tasks, examples, instructions, choices, hypothetical work scenarios, statements from the other party, counterarguments, additional conditions, documents, images, audio, or video. Images, audio, or video may be output via a display unit, audio output unit, or video output unit, etc.

[0051] Specific examples of the information presented will be explained for each type of exercise. For example, in a scenario equivalent to an in-basket exercise, the presented information may include a hypothetical work scenario in which the person being evaluated is assumed to be a newly appointed manager and must handle multiple cases within a limited time. For example, the presented information may include instructions such as, "You are the newly appointed department head. There are several unprocessed cases on your desk. Please decide on a course of action within the limited time," along with an email or report showing the multiple cases. The computer may generate multiple unprocessed cases based on the position assigned to the person being evaluated, their decision-making authority, the organizational structure, or the positions of those involved.

[0052] In another example, in a scenario equivalent to a policy-making exercise or an analytical presentation exercise, the information provided may include a task that requires students to analyze multiple documents and formulate a policy. For example, the information provided may include an instruction such as, "Based on the following documents, formulate a business policy for the next term and explain its rationale," along with a document showing multiple numerical data.

[0053] In another example, in a scenario equivalent to an interview exercise, the presented information may include statements, questions, counterarguments, or additional conditions from the other party. For example, the presented information may include a counterargument to the person being evaluated, such as, "Doesn't that policy place too much burden on the team?" or an additional condition, such as, "The budget has been cut in half." Furthermore, the presented information may include a dilemma situation that asks which of two conflicting demands to prioritize, for example, "If you prioritize quality, you won't meet the deadline; if you prioritize the deadline, quality will suffer. Which will you prioritize, and how?"

[0054] In another example, in a scenario equivalent to a group discussion exercise, the presented information may include statements from multiple participants and tasks that encourage the person being evaluated to participate in the discussion. In yet another example, in a scenario equivalent to an essay or writing exercise, the presented information may include instructions for the person being evaluated to write about a given topic. In one example, in a scenario equivalent to an interview exercise or group discussion exercise, a dialogue takes place between the person being evaluated and a participant, and the participant's statements, facial expressions, or voice may be changed in response to the person being evaluated.

[0055] According to the embodiment, by generating information to be presented to the person being evaluated based on the evaluation items and the second criterion, information to be presented with content or expression corresponding to the evaluation items is output, making it easier to obtain the information necessary for evaluation regarding those evaluation items. In particular, by including at least one of the following in the information presented to the person being evaluated: questions, tasks, examples, instructions, choices, hypothetical work scenarios, statements from the other party, counterarguments, additional conditions, documents, images, audio, or video, a variety of information formats can be presented according to the evaluation items, making it easier to obtain the information necessary for evaluation regarding those evaluation items.

[0056] Next, the acquisition of response information will be explained. In one example, the computer may acquire text information, audio information or text information generated from such audio, information indicating facial expressions, etc., and information indicating response time, and record these as response information. The response information may include at least one of the following: text information entered by the person being evaluated, audio information indicating the person being evaluated's speech, text information generated from such speech, information indicating the person being evaluated's facial expressions, posture, or gaze, and information indicating the response time to the information presented to the person being evaluated. Here, audio information indicating the person being evaluated's speech and text information generated from such speech are different types of response information, and the computer may acquire and record both.

[0057] Specific examples of acquiring and recording response information are described below. For example, a computer may record character information entered by the person being evaluated as text acquired via an input unit. A computer may record speech information representing the person being evaluated as waveform or encoded speech data acquired via a speech processing unit, and record character information generated from that speech as the result of recognition by a speech recognition unit. A computer may record information representing the person being evaluated's facial expression, posture, or gaze as information representing features extracted from captured video by an image processing unit or image recognition unit. A computer may record information indicating response time as information indicating the time from the output of the presented information to the acquisition of the response or the time required for the response.

[0058] In another example, the computer may acquire and record nonverbal information such as the person being evaluated nodding or giving verbal cues, or information indicating the volume, pitch, speed, intonation, or length of silence of their speech, as response information. It should be noted that this information does not directly represent the person being evaluated's inner state, but rather is treated as information indicating characteristics extracted by the computer from the captured video or audio.

[0059] According to the embodiment, multiple types of response information are acquired, including not only textual information but also audio information, textual information generated from the audio, information indicating facial expressions, posture or gaze, or information indicating response time. By applying evaluation criteria to these, both verbal and nonverbal information can be acquired, making it easier to evaluate the characteristics of the person being evaluated from multiple perspectives.

[0060] Figure 2 shows an example of a processing flow that applies evaluation criteria to response information and outputs evaluation values ​​for the person being evaluated. In this example, the computer obtains evaluation criteria corresponding to the evaluation items (S201), applies the evaluation criteria to the response information (S202), outputs evaluation points corresponding to the evaluation items (S203), and outputs additional information showing the relationship between the response information and the evaluation items (S204).

[0061] Outputting an evaluation value may include outputting an evaluation score corresponding to an evaluation item and at least one of additional information, based on the result of applying the evaluation criteria to the response information. In other words, the computer is not limited to outputting both the evaluation score and the additional information, but may output either the evaluation score or the additional information. For example, the computer may output the evaluation score if it can be output from the response information, and output additional information if it cannot immediately output the evaluation score from the response information.

[0062] The supplementary information is information that shows the relationship between the response information and the evaluation item. For example, the supplementary information may include at least one of the following: the response information does not match the evaluation item; the response information lacks sufficient information to evaluate the evaluation item; and the response information relates to an evaluation item other than the evaluation item. Based on the supplementary information, the computer may change the information it presents to the person being evaluated next, as described later.

[0063] This section describes a specific example of outputting evaluation scores. In one example, the evaluation criteria for the evaluation item "Analysis" include multiple elemental concepts, priority levels (weights) corresponding to each elemental concept, multiple scores, and behavioral examples corresponding to each score. The computer may extract information indicating claims, reasons, judgment criteria, priorities, or scope from the evaluator's response information and associate these with behavioral examples to output evaluation scores corresponding to the scores. Furthermore, the computer may integrate the evaluation scores for each elemental concept based on the priority levels (weights) of each elemental concept to output the evaluation score for the evaluation item "Analysis". The application of evaluation criteria may be performed by keyword matching, classification processing, scoring, rule-based processing, inference by an AI model, or a combination thereof.

[0064] Specific examples of outputting additional information are described below. For example, if the evaluator's response information consists entirely of content that differs from the evaluation items asked in the presented question or task, the computer may output additional information indicating that the response information does not match the evaluation items. For example, if the computer does not extract evaluation grounds corresponding to the evaluation items from the response information and there is insufficient information to output an evaluation score, the computer may output additional information indicating that there is insufficient information in the response information to make an evaluation. For example, if the response information includes content related to another evaluation item, "creativity," rather than the presented evaluation item "collaboration," the computer may output additional information indicating that the response information is related to the other evaluation item.

[0065] According to the embodiment, by outputting at least one of an evaluation score corresponding to an evaluation item and additional information based on the result of applying the evaluation criteria to the response information, an evaluation score is output for responses for which an evaluation score can be output, and for responses for which an evaluation score cannot be output immediately, the relationship between the response information and the evaluation item is output as additional information, making it easier to use for modifying the information presented next.

[0066] According to the embodiment, the additional information includes at least one of the following: the response information does not match the evaluation item; the response information lacks sufficient information to evaluate the evaluation item; and the response information relates to an evaluation item other than the evaluation item. This makes it easier to modify the information presented below according to the reason, even for responses for which an evaluation score cannot be immediately obtained.

[0067] An example of the data structure for evaluation items and evaluation criteria will be described. In the first embodiment, the evaluation system has a hierarchical structure consisting of multiple evaluation axes and multiple evaluation items included in each evaluation axis. In one example, four evaluation axes are given: categories related to attitude, categories related to interpersonal relationships, categories related to thinking, and categories related to work performance, and each evaluation axis includes multiple evaluation items. Hereinafter, the names of the evaluation axes may be abbreviated, and the category related to attitude may be shown as evaluation axis "attitude", the category related to interpersonal relationships and relationship formation as evaluation axis "interpersonal relationships", the category related to thinking and cognition as evaluation axis "thinking", and the category related to work performance as evaluation axis "work performance".

[0068] For example, the evaluation axis "Attitude" includes proactiveness, activity, persistence, and stability; the evaluation axis "Interpersonal" includes influence, collaboration, empathy, and support; the evaluation axis "Thinking" includes analysis, integration, development, and creativity; and the evaluation axis "Job Performance" includes objective setting, planning, human resource utilization, and control. The evaluation axes and evaluation items are hierarchical and not synonymous. The number and names of evaluation items included in each evaluation axis may vary depending on the application and are not limited to these examples.

[0069] Each evaluation item may have information indicating a definition, concept, keywords representing the concept, elemental concepts, priority (weight) of the elemental concepts, score, and at least one example of behavior corresponding to the score. For example, the evaluation criteria may have a tabular data structure for each evaluation item that associates multiple elemental concepts, priority (weight) corresponding to each elemental concept, multiple scores (for example, multiple levels such as high level, standard level, and insufficient level), and example behaviors corresponding to each elemental concept and each score.

[0070] This section explains an example of the evaluation item "cooperation," which belongs to the evaluation axis "interpersonal relationships." The evaluation item "cooperation" is defined as the ability to cooperate willingly in step with those around you, and may have elemental concepts such as "communication," "friendliness," or "collaboration." For the elemental concept "communication," an example of behavior corresponding to a high score may be "aligning the understanding of the situation with stakeholders through reporting or communication," while an example of behavior corresponding to a low score may be "proceeding unilaterally without sharing with stakeholders."

[0071] The computer may output evaluation scores corresponding to the evaluation items by associating information indicating the behavior of the person being evaluated, extracted from the response information, with examples of these behaviors. Furthermore, the computer may output an evaluation score for the evaluation item "cooperation" by integrating the evaluation scores for each element concept based on the priority (weight) of each element concept. The specific number of evaluation levels, the values ​​of the priority (weight) of the element concepts, or the thresholds may be determined as appropriate depending on the operation.

[0072] In the second embodiment, the evaluation system may have a hierarchical structure consisting of multiple higher-level categories, multiple skill sets included in each higher-level category, and multiple behaviors corresponding to each skill set. For example, the higher-level categories may include categories related to communication, leadership, adaptability, interpersonal relationships, task management, results production, subordinate development, and self-development, and each higher-level category may include multiple skill sets. Multiple behaviors corresponding to each skill set may be associated with that skill set.

[0073] In the second embodiment as well, the computer may treat the actions corresponding to each skill set as evaluation criteria, similar to the action examples in the first embodiment, apply them to the response information to output evaluation values, and update the degree. That is, in the first embodiment, the evaluation items may correspond to evaluation items belonging to the evaluation axis, and in the second embodiment, they may correspond to skill sets belonging to higher categories or actions corresponding to said skill sets. The information processing method according to the embodiment can be applied to either the evaluation system of the first or second embodiment, and is not limited to a specific evaluation system, but can be applied to other evaluation systems as well.

[0074] Next, we will explain how to update the extent to which the information necessary for evaluation has been obtained. For example, the computer may obtain the amount of response information, the number of evaluation criteria, whether or not there is any unobtained information, and the confidence level of the evaluation value, and update the extent based on these. The extent may be updated based on at least one of the following: the amount of response information obtained for the evaluation item, the number of evaluation criteria obtained for the evaluation item, whether or not there is any unobtained information for the evaluation item, or the confidence level of the evaluation value.

[0075] In one example, the computer may manage, for each evaluation item, the number of evaluation criteria obtained, the type of response information, the number of times it has been presented, the specificity of the response, whether there are any inconsistencies, and the confidence level or remaining amount of the evaluation value, and update the degree of each evaluation item based on these. In another example, the computer may maintain the degree for each evaluation item as a state indicating one of several levels. For example, the computer may indicate that an evaluation item with fewer than a predetermined number of evaluation criteria is in a "insufficient" state, and that an evaluation item with more than the predetermined number of evaluation criteria and a confidence level of the evaluation value is above a predetermined value is in a "sufficient" state. The computer may identify evaluation items that indicate a "insufficient" degree as candidates for the next evaluation item to be evaluated.

[0076] Let's explain some specific examples of updating the level of evaluation. For example, regarding the evaluation item "collaboration," if the person being evaluated makes a response that mentions collaboration with stakeholders, and multiple evaluation criteria are extracted from that response, the computer may update the level of the evaluation item "collaboration" to indicate a higher level. On the other hand, regarding the evaluation item "creativity," if no evaluation criteria are extracted from the person being evaluated's response, the computer may maintain the level of the evaluation item "creativity" at a low level.

[0077] According to the embodiment, by updating the extent to which the information necessary for evaluation has been acquired for each evaluation item, it becomes easier to grasp the status of information acquisition for each evaluation item, and it becomes easier to suppress the omission of evaluation items with insufficient information. In particular, by updating the extent based on at least one of the following: the amount of response information acquired, the number of evaluation basis items, the presence or absence of unacquired information, or the confidence level of the evaluation value, it becomes easier to grasp the status of information acquisition for each evaluation item from multiple perspectives and to identify evaluation items with insufficient information.

[0078] Figure 3 shows an example of a processing flow for generating and outputting confirmation items, etc., for evaluation items where the extent to which the information necessary for evaluation has been acquired does not meet the predetermined conditions. In this example, the computer acquires the extent to which the information necessary for evaluation has been acquired (S301), identifies evaluation items that do not meet the predetermined conditions (S302), generates confirmation items, etc. (S303), and outputs this as the information to be presented next (S304).

[0079] Modifying the information presented to the person being evaluated may include generating and outputting questions, tasks, objections, additional conditions, or confirmations corresponding to evaluation items where the extent to which necessary information for evaluation has been obtained does not meet the predetermined conditions. The predetermined conditions may include, for example, at least one of the following: the degree is below a threshold, the number of evaluation grounds is below a predetermined number, there is unobtained information, or the confidence level of the evaluation value is below a predetermined value. The specific threshold or predetermined number may be determined as appropriate depending on the operation.

[0080] This section explains specific examples of how to address evaluation items with insufficient information. For example, if the level of the evaluation item "Control" is below the threshold, the computer may generate a confirmation question such as, "How will you respond when the plan and actual results diverge?" and output it as the next piece of information. In another example, if the level of the evaluation item "Planning" is below the threshold, the computer may generate and output a task such as, "What procedures and schedule will you use to implement that policy?"

[0081] According to the embodiment, by generating and outputting questions, issues, counterarguments, additional conditions, or confirmations corresponding to evaluation items where the extent to which necessary information for evaluation has been acquired does not meet predetermined conditions, it becomes easier to acquire additional information necessary for evaluation items with insufficient information and to reduce the likelihood of missing evaluation items.

[0082] Figure 4 shows an example of a processing flow for generating and outputting information that makes the claims and other statements contained in the response information more specific. In this example, the computer acquires the response information (S401), extracts information that indicates the claims and other statements (S402), generates information that encourages concretization (S403), and outputs this as the next information to be presented (S404).

[0083] Modifying the information presented to the person being evaluated may include generating and outputting information that makes the claims, reasons, criteria, or grounds more specific, based on the information that indicates the claims, reasons, criteria, or grounds included in the response information.

[0084] Let's explain some specific examples of in-depth questioning. For example, if the person being evaluated makes a response that includes the assertion, "I will process the highest priority cases first," the computer may extract information that supports this assertion and generate information that prompts further clarification, such as, "What criteria will you use to determine priority?" or "Which specific cases will you process first?" and output this information as the next piece of information to be presented. In another example, if the person being evaluated responds with only a conclusion without explaining the reasons, the computer may generate and output information that prompts further clarification, such as, "Please explain the reasons for your decision."

[0085] According to the embodiment, by generating and outputting information that makes the claims, reasons, judgment criteria, or grounds included in the response information more specific, it becomes possible to delve deeper into the judgment criteria or grounds of the person being evaluated and to more accurately evaluate the characteristics of the person being evaluated.

[0086] Next, we will describe another example of a process that modifies the information presented to the person being evaluated. In one example, if the computer detects a contradiction or inconsistency between the information contained in the response information and other response information already obtained from the person being evaluated or information already presented to the person being evaluated, it may generate and output information requesting confirmation regarding the contradiction or inconsistency. For example, if the person being evaluated states in a previous response, "I will proceed while respecting the opinions of those on the ground," and in a later response, "I will make a decision without consulting those on the ground," the computer may detect an inconsistency between the two responses and generate and output information requesting confirmation such as, "You said earlier that you would respect the opinions of those on the ground, but is it correct to understand that you will make a decision without consulting them?" This makes it easier to obtain the information necessary to evaluate the consistency of the person being evaluated's judgment or their understanding of the underlying assumptions.

[0087] In another example, the computer may select evaluation items from among several evaluation items to which the degree to which the necessary information for evaluation has been obtained is relatively low, or to evaluation items that have a high priority in relation to the amount of interaction remaining with the person being evaluated, and generate and output information corresponding to the selected evaluation items. In this case, the computer may obtain multiple evaluation items, identify evaluation items with a relatively low degree of completion, obtain the remaining amount, identify evaluation items with a high priority, select evaluation items, and generate and output information corresponding to the selected evaluation items.

[0088] In yet another example, changing the information presented to the person being evaluated may include selecting the next evaluation item from among several evaluation items based on the amount of interaction remaining with the person being evaluated, i.e., the remaining time allocated to the evaluation of the person being evaluated, or the number of questions or tasks remaining for the person being evaluated out of a predetermined number of questions or tasks, and generating and outputting information corresponding to the selected evaluation item. In one example, the computer may obtain the number of questions or tasks remaining for the person being evaluated, and if the number of remaining questions or tasks is less than a predetermined value, it may select an evaluation item that allows observation of multiple evaluation items with a single piece of presented information.

[0089] According to the embodiment, by generating information that selects and presents evaluation items for which the degree to which the necessary information for evaluation has been obtained is relatively low, or evaluation items that have a high priority relative to the remaining amount, it becomes easier to prioritize the evaluation of evaluation items for which the necessary information for evaluation is insufficient, under the constraints of a limited time or number of attempts.

[0090] Next, we will explain the process of reflecting the tendency or intensity of expression in the information presented. The tendency and intensity of expression handled in this process are values ​​(labels) that indicate the expression conditions set by the computer in the presented information, and do not indicate the inner state of the computer or AI model. For example, the computer may acquire the tendency or intensity of expression, set expression conditions in the presented information, reflect them in the wording, etc., and output this as presented information. The information presented to the person being evaluated may include information indicating the tendency or intensity of expression, and such tendency or intensity of expression may be reflected in at least one of the wording, voice, facial expression, posture, gaze, or response interval.

[0091] For example, the computer may maintain the subject of the question, the work setting, or the evaluation items being assessed, while only changing the tendency of expression (a value indicating the expression condition such as positive, negative, neutral, skeptical, confused, argumentative, or calm) and intensity (multiple levels in an example).

[0092] Specific examples of reflecting the tendency or intensity of expression are described below. For example, if the other party is set to a value indicating a skeptical expression condition, the computer may generate the wording of the presented information as "Will that policy really work?" If the other party is set to a value indicating a neutral expression condition, the computer may generate the wording of the presented information as "Please explain that policy a little more." Also, if the value is set to an expression condition indicating a high intensity, the computer may reflect this by increasing the volume of the other party's voice or shortening the response interval. In another example, the computer may determine the tendency or intensity of expression according to the evaluation item being evaluated. For example, when evaluating the evaluation item "stability," the computer may set the tendency of the other party's expression to a counter-argumentary value and set the intensity to a high level.

[0093] According to the embodiment, by determining the tendency or intensity of the expression of the information presented according to the evaluation items to be evaluated, a situation is created in which the information necessary for evaluation of the evaluation items is easily revealed, making it easier to evaluate the characteristics of the person being evaluated.

[0094] Next, we will describe the process of executing an information processing method by referring to the information stored in the AI ​​model. In one example, the computer may acquire information related to the AI ​​model, refer to training data, refer to prompts, refer to output formats, and execute an information processing method by referring to these. The information stored in the AI ​​model may include at least one of the following: training data, training data, parameters of the trained model, prompts, information indicating procedures, information indicating evaluation procedures, information indicating output formats, reference information, or information indicating validation rules.

[0095] Two forms of memory applied to an AI model are described below. In the first form, the memory applied to the AI ​​model may be implemented as a trained model learned based on training data that associates the evaluator's response information with the evaluation results. For example, the AI ​​model may be learned based on training data that associates past evaluator response information with the evaluation values ​​assigned to that response information, and the computer may use the AI ​​model to output evaluation values ​​for new response information. In the second form, the memory applied to the AI ​​model may be referenced during inference as prompt or procedure information that includes information indicating evaluation criteria, exercise type, response history, reference information, output format, or validation rules. For example, the computer may provide the AI ​​model with prompt or procedure information that includes information indicating selected evaluation items, evaluation criteria corresponding to those evaluation items, and output format, in order to obtain presented information or evaluation values.

[0096] This section describes specific examples of input / output and verification rules applied to AI models. In one example, a computer may input information indicating evaluation items, evaluation criteria, response information, presentation history, response history, remaining quantity, degree, or output format into an AI model, retrieve information output from the AI ​​model, and generate evaluation values, evaluation justification, additional presentation information, or information requesting confirmation. The computer may verify the information output from the AI ​​model based on verification rules in terms of output format, correspondence with evaluation items, consistency with evaluation criteria, or consistency with previously presented information. If the verification results in the information not meeting these rules, the computer may input the information back into the AI ​​model and retrieve it again.

[0097] According to the embodiment, by referencing the information stored in the AI ​​model to generate presentation information or output evaluation values, consistency in evaluation can be more easily ensured. In particular, if the information stored in the AI ​​model includes at least one of the following: training data, teacher data, parameters of the trained model, prompts, information indicating procedures, information indicating evaluation procedures, information indicating output formats, reference information, or information indicating verification rules, then referencing this information to generate presentation information or output evaluation values ​​can be more easily used to ensure consistency in evaluation.

[0098] Figure 5 shows an example of the hardware configuration of a computer according to an embodiment. In this example, the computer includes a CPU 1201, a ROM 1202, a RAM 1203, a storage unit 1204, an input interface 1205, a display interface 1206, a communication interface 1207, an external memory interface 1208, an input unit 1216, and a display unit 1217. The computer may communicate with external devices via a communication network 1215, or it may read information from a storage medium 1218.

[0099] The information processing method according to the embodiment is a method in which information processing by software is specifically realized using the above-mentioned hardware resources. Each process in Figure 1 described above can be realized by the CPU 1201 reading and executing a program stored in the storage unit 1204, in cooperation with hardware resources such as the RAM 1203, input unit 1216, display unit 1217, communication interface 1207, or the AI ​​processing unit described later. In one example, the input unit 1216 acquires character information entered by the person being evaluated, and the display unit 1217 displays the information to be presented. The communication interface 1207 may send and receive information to and from a terminal or external device used by the person being evaluated via the communication network 1215. The external memory interface 1208 may read a program or information indicating evaluation criteria from the storage medium 1218. The RAM 1203 may temporarily hold response information, evaluation values, or information indicating the degree used during processing. The storage unit 1204 may store evaluation items, evaluation criteria, a first criterion, a second criterion, response information, evaluation values, or information indicating the degree.

[0100] Figure 6 shows an example of a functional configuration according to an embodiment. In one example, the computer comprises an AI processing unit 1310, a speech recognition unit 1311, an image recognition unit 1312, a speech synthesis unit 1313, and a video output unit 1314 as its functional configuration. The computer may be connected to, or include, a speech processing unit 1302, an image processing unit 1303, a speech output unit 1304, a display unit 1305, a storage unit 1306, a communication unit 1307, or an external device 1308. Note that the configuration of each part shown in Figure 6 is just an example and is not limited to these.

[0101] Specific examples of the operation of each part are described below. In one example, the AI ​​processing unit 1310 takes evaluation items and evaluation criteria as input and outputs presentation information, evaluation values, or information requesting confirmation. The speech recognition unit 1311 generates text information from speech information representing the speech of the person being evaluated, and the image recognition unit 1312 extracts information representing the facial expression, posture, or gaze of the person being evaluated from the captured video. The speech synthesis unit 1313 generates speech representing the speech of the other party from the wording of the presentation information, and the video output unit 1314 outputs the video of the other party. The computer may control the processing order of each of these parts. In one example, the computer may output the video of the other party or the questioner as an avatar to the display unit, and the information presented to the person being evaluated may be presented via the avatar.

[0102] A specific example of the processing steps is described below. In one example, the computer causes the AI ​​processing unit 1310 to generate presentation information based on selected evaluation items, the speech synthesis unit 1313 to generate the audio of the presentation information, and the video output unit 1314 to output the video of the other person. In response to the person being evaluated, the speech processing unit 1302 acquires the audio information, the speech recognition unit 1311 generates the text information from the audio information, the image processing unit 1303 acquires the captured video, and the image recognition unit 1312 extracts information indicating facial expressions, posture, or gaze from the video. The AI ​​processing unit 1310 may also apply evaluation criteria to this response information and output an evaluation value.

[0103] Next, variations will be described. In one example, the computer may act as the interlocutor, presenter, and evaluator, and directly interact with the person being evaluated. In another example, if the computer receives input indicating a correction to the evaluation value it has output, the result of that correction may be reflected in the training data or reference information. In yet another example, the computer may output a portion of the evaluation results of multiple people being evaluated for verification by a human assessor.

[0104] Let's describe another variation. The information and responses presented to the person being evaluated may, for example, be used in an in-basket exercise, a policy-making exercise or an analytical presentation exercise, an interview exercise, a group discussion exercise, or an essay or writing exercise. The computer may select the type of exercise according to the evaluation items it wants to observe, or it may identify the evaluation items that can be observed according to the selected type of exercise. When changing the information presented to the person being evaluated next, the computer may prioritize evaluation items that can be observed in the exercise currently being conducted as the target of the next measurement. This prevents an exercise from being interrupted midway and allows the evaluation to proceed while reducing the burden on the person being evaluated.

[0105] Let's consider another variation. In one example, the computer may generate a value indicating the intensity of the response or psychological burden based on information showing the evaluator's response delay, silence, speech rate, volume, facial expression, gaze, or posture. Based on this value, the computer may increase or decrease the intensity of the presentation of the information it presents, perform a process to stop the presentation, or output information requesting human confirmation. It should be noted that the value indicating the intensity of the response or psychological burden, as used herein, is a value based on features extracted by the computer from the evaluator's response information and does not represent the evaluator's inner state itself.

[0106] Let's consider another variation. In one example, the computer may generate an overall evaluation by integrating the evaluation values ​​for each evaluation item based on weights. The overall evaluation may include aggregations for each evaluation axis or aggregations with confidence levels. In one example, the evaluation results may be output as information for the individual, information for the organization, information for training design, or information for a human assessor. In another example, the computer may output, along with the evaluation values, information indicating the basis for the evaluation, including some of the response information used to generate the evaluation values, information indicating characteristics obtained from the response information, and information indicating the correspondence with the evaluation items.

[0107] Let's describe another variation. In one example, the computer may switch between the evaluation system according to the first embodiment and the evaluation system according to the second embodiment, or select and use either one. In yet another example, the computer may use an evaluation system different from these, and the number, names, or hierarchical structure of the evaluation items may be determined as appropriate depending on the operation. According to the embodiment, by outputting evaluation values ​​to multiple people being evaluated based on common evaluation items, common evaluation criteria, or standardized conditions, it becomes easier to suppress variations in evaluation results. Furthermore, with a configuration in which the terminals used by the people being evaluated are connected via a communication network, evaluations can be performed without requiring the people being evaluated to gather in one place, making it easier to reduce the burden required for evaluation.

[0108] The information processing method according to the embodiment may be implemented as a program that causes a computer to execute the information processing method. The program may also be provided by recording it on a computer-readable recording medium. The recording medium referred to here is a non-transitory recording medium and does not contain the transient propagating signal itself. For example, the program may be recorded on the storage medium 1218 in Figure 5, read via the external memory interface 1208, and stored in the storage unit 1204. The program may cause the computer to execute at least a portion of each step in Figure 1, and each of the processes shown in Figures 2 to 4.

[0109] According to the embodiment, by recording and providing the program on a recording medium, or by having a computer execute each process, it becomes easier to implement an information processing method that efficiently obtains appropriate evaluations for each person being evaluated on various computers. [Explanation of Symbols]

[0110] 1201 CPU 1202 ROM 1203 RAM 1204 Storage section 1205 Input Interface 1206 Display Interface 1207 Communication Interface 1208 External memory interface 1215 Communication Network 1216 Input section 1217 Display section 1218 Storage medium 1302 Audio Processing Unit 1303 Image Processing Unit 1304 Audio output section 1305 Display section 1306 Storage section 1307 Communications Department 1308 External device 1310 AI Processing Unit 1311 Voice Recognition Unit 1312 Image Recognition Unit 1313 Speech Synthesis Unit 1314 Video Output Section

Claims

1. An information processing method in which a computer operating in conjunction with an AI model outputs evaluation values ​​for the characteristics of a person being evaluated, The method involves selecting from among multiple evaluation items the evaluation items used to evaluate the person being evaluated are determined by obtaining the extent to which the multiple evaluation items are covered, the amount of interaction remaining with the person being evaluated, and the degree to which the information necessary for evaluation has been obtained for each of the multiple evaluation items, and then selecting based on a first criterion based on the obtained extent to which the coverage, the amount remaining, and the degree, wherein the degree is a value or state determined for each of the multiple evaluation items according to the reliability of the evaluation value. The selected evaluation items and the second criteria based on the acquired degree are input into the AI ​​model, and the information to be presented to the person being evaluated, which is output from the AI ​​model, is acquired and output. To obtain response information to the information presented to the person being evaluated, The evaluation criteria corresponding to the evaluation items, the evaluation items themselves, and the response information are input to the AI ​​model, and the AI ​​model applies the evaluation criteria to the response information and outputs the evaluation value of the person being evaluated. Based on the response information, the degree of the evaluation item is updated, The updated degree and the response information are input to the AI ​​model, the information output from the AI ​​model is obtained, and the information to be presented to the person being evaluated is then changed. It has, An information processing method wherein the AI ​​model is equipped with storage for at least one of the evaluation items, the information to be presented to the person being evaluated, the evaluation criteria, the first criterion, the second criterion, or information indicating the response information.

2. The selection based on the first criterion is carried out so as to cover all of the evaluation items for the person being evaluated. The information processing method according to claim 1.

3. The information processing method according to claim 2, wherein the selection based on the first criterion includes determining one evaluation item that has not yet been covered if the remaining amount is greater than or equal to a predetermined value.

4. The information processing method according to claim 1, wherein the remaining amount is the remaining time out of a predetermined amount of time for interaction with the person being evaluated, or the remaining number of interactions out of a predetermined number of interactions with the person being evaluated.

5. The information processing method according to claim 1, wherein the second criterion is further determined based on the evaluation value based on the response information.

6. The information processing method according to claim 1, wherein the information presented to the person being evaluated includes at least one of questions, tasks, examples, instructions, choices, a hypothetical work scenario, statements from an opposing party, counterarguments, additional conditions, documents, images, audio, or video.

7. The information processing method according to claim 1, wherein the response information includes at least one of the following: character information entered by the person being evaluated, audio information indicating the speech of the person being evaluated, character information generated from the speech, information indicating the facial expression, posture or gaze of the person being evaluated, and information indicating the response time to the information presented to the person being evaluated.

8. Obtaining and outputting the aforementioned evaluation value is, Based on the results of applying the evaluation criteria to the response information, for responses that can output an evaluation score, the evaluation value including the evaluation score corresponding to the evaluation item is output, and for responses that cannot immediately output an evaluation score, additional information is output. The aforementioned additional information is information that shows the relationship between the response information and the evaluation item. The information processing method according to claim 1.

9. The aforementioned additional information is, The information processing method according to claim 8, comprising at least one of the following: the response information does not match the evaluation item; the response information lacks sufficient information to evaluate the evaluation item; and the response information relates to an evaluation item other than the evaluation item.

10. Changing the information presented to the person being evaluated means The information processing method according to claim 1, further comprising generating and outputting questions, issues, counterarguments, additional conditions, or confirmations corresponding to evaluation items to which the extent to which the information necessary for the evaluation has been acquired does not meet the predetermined conditions.

11. Changing the information presented to the person being evaluated means The information processing method according to claim 1, further comprising generating and outputting information that makes the claims, reasons, criteria or grounds more specific, based on the information that indicates the claims, reasons, criteria or grounds contained in the response information.

12. The information processing method according to claim 1, wherein the information presented to the person being evaluated includes information indicating the tendency or intensity of expression, and the tendency or intensity of expression is reflected in at least one of wording, voice, facial expression, posture, gaze, or response interval.

13. The information processing method according to claim 1, wherein the information stored in the AI ​​model includes at least one of training data, teacher data, parameters of the trained model, prompts, information indicating procedures, information indicating evaluation procedures, information indicating output formats, reference information, or information indicating verification rules.

14. A program that causes a computer to execute the information processing method described in any one of claims 1 to 13.

Images