Data processing device, data processing method, program, and data processing system

Abstract

To enable evaluation of a participant of interview according to contents of utterance of the participant.SOLUTION: A data processing device 1 includes: a data acquisition unit 131 which acquires conversation text data indicating contents of conversation in an interview performed by an applicant and an interviewer; a label assignment unit 132 which assigns a label indicating an attribute of contents of each partial text data to each of a plurality of pieces of partial text data contained in the conversation text data by executing a natural language process to the conversation text data; a data creating unit 133 which creates conversation structured data being composed of the plurality of labels corresponding to the plurality of pieces of partial text data in the conversation text data and indicating the structure of the conversation; and an evaluation unit 134 which evaluates at least one of the applicant and the interviewer on the basis of patterns of the plurality of labels contained in the conversation structured data.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a data processing device, a data processing method, a program, and a data processing system for assisting in the evaluation of interview participants. [Background technology]

[0002] BACKGROUND ART Conventionally, a system is known that evaluates an applicant by analyzing a video of the job interview (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7333404 Summary of the Invention [Problem to be solved by the invention]

[0004] It is difficult to accurately evaluate a person's abilities based on appearance alone, so there is a need to be able to evaluate participants based on what they say during an interview.

[0005] The present invention has been made in consideration of these points, and aims to make it possible to evaluate participants in an interview based on the content of their statements. [Means for solving the problem]

[0006] A data processing device of a first aspect of the present invention comprises a data acquisition unit that acquires conversation text data indicating the content of a conversation between an applicant and an interviewer during an interview; a label assignment unit that performs natural language processing on the conversation text data to assign labels indicating the attributes of the content of each of a plurality of partial text data included in the conversation text data; a data creation unit that creates conversation structured data that indicates the structure of the conversation and is composed of a plurality of labels corresponding to a plurality of the partial text data in the conversation text data; and an evaluation unit that evaluates at least one of the applicant and the interviewer based on the patterns of the plurality of labels included in the conversation structured data.

[0007] The evaluation unit may evaluate at least one of the applicant and the interviewer by comparing the conversation structured data with reference structured data that serves as a standard for evaluating the conversation and includes multiple labels.

[0008] The evaluation unit may evaluate at least one of the applicant and the interviewer based on the content of the differences between the patterns of the multiple labels in the conversation structured data and the patterns of the multiple labels in the reference structured data.

[0009] The data processing device may further have a memory unit that stores a plurality of the standard structured data associated with each of a plurality of evaluation items for evaluating the applicant or the interviewer, the data creation unit creates the conversation structured data associated with each of the plurality of evaluation items, and the evaluation unit may evaluate at least one of the applicant and the interviewer by comparing the conversation structured data associated with the target evaluation items for evaluating the applicant or the interviewer with the standard structured data associated with the target evaluation items.

[0010] The labels may include a first label indicating an attribute that the question has an abstract content and a second label indicating an attribute that the question has a specific content, and the evaluation unit may give a higher evaluation to the interviewer the more times there are transitions between the interviewer's question corresponding to the first label and the interviewer's question corresponding to the second label.

[0011] The evaluation unit may evaluate at least one of the applicant and the interviewer based on the number of predetermined labels included in the conversation structured data. The evaluation unit may evaluate the interviewer higher the number of predetermined labels corresponding to the attribute that the applicant's statement is a statement that the interviewer affirms. The evaluation unit may evaluate the interviewer higher the number of predetermined labels corresponding to the attribute that the applicant's statement is a statement that the interviewer explains why the interviewer affirms the applicant's statement.

[0012] The evaluation unit may also evaluate the interviewer higher the more the number of specified labels corresponding to the attribute that the statement explains the relevance between the applicant's statement and the work of the organization to which the interviewer belongs.

[0013] The evaluation unit may also evaluate the interviewer higher the more the number of specified labels corresponding to the attribute that the interviewer's statement explains the appeal of the business or job at the organization to which the applicant is applying.

[0014] The evaluation unit may increase the evaluation of the interviewer as the number of predetermined labels corresponding to the attribute that the statement indicates the subjective opinion of the interviewer increases. The evaluation unit may increase the evaluation of the interviewer as the number of predetermined labels corresponding to the attribute that the statement includes a specific explanation increases.

[0015] The evaluation unit may evaluate the interviewer higher when a plurality of the labels included in the conversation structured data corresponding to the applicant satisfy a predetermined condition than when the predetermined condition is not satisfied.

[0016] The evaluation unit may evaluate the applicant higher when the conversation structured data corresponding to the applicant satisfies an evaluation criterion regarding specificity than when the evaluation criterion is not satisfied.

[0017] The evaluation unit may determine an evaluation value of the applicant for each of a plurality of indicators for evaluating the applicant based on a plurality of label patterns corresponding to each of the plurality of indicators.

[0018] A data processing method of a second aspect of the present invention includes the steps of: acquiring conversation text data indicating the content of a conversation during an interview between an applicant and an interviewer, executed by a computer; assigning labels indicating attributes of the content of each of a plurality of partial text data included in the conversation text data by performing natural language processing on the conversation text data; creating conversation structured data indicating the structure of the conversation, which is composed of a plurality of labels corresponding to a plurality of the partial text data in the conversation text data; and evaluating at least one of the applicant and the interviewer based on the patterns of the plurality of labels included in the conversation structured data.

[0019] A third aspect of the present invention provides a program for causing a computer to execute the following steps: acquiring conversation text data indicating the content of a conversation between an applicant and an interviewer during an interview; performing natural language processing on the conversation text data to assign labels indicating the attributes of the content of each of the partial text data to a plurality of partial text data included in the conversation text data; creating conversation structured data that indicates the structure of the conversation and is composed of a plurality of labels corresponding to a plurality of the partial text data in the conversation text data; and evaluating at least one of the applicant and the interviewer based on the patterns of the plurality of labels included in the conversation structured data.

[0020] A fourth aspect of the present invention provides a data processing system comprising a data processing device and an information terminal, wherein the data processing device comprises: a data acquisition unit that acquires conversation text data indicating the content of a conversation between an applicant and an interviewer during an interview; a label assignment unit that performs natural language processing on the conversation text data to assign labels indicating the attributes of the content of each of the partial text data to a plurality of partial text data included in the conversation text data; a data creation unit that creates conversation structured data that indicates the structure of the conversation and is composed of a plurality of labels corresponding to a plurality of the partial text data in the conversation text data; and an evaluation unit that evaluates at least one of the applicant and the interviewer based on the patterns of the plurality of labels included in the conversation structured data, and the information terminal transmits information for identifying the applicant or the interviewer to the data processing device, and thereby obtains from the data processing device the results of the evaluation of the applicant or the interviewer corresponding to the transmitted information. [Effects of the Invention]

[0021] The present invention has the effect of making it possible to evaluate participants in an interview based on the content of their comments. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is a diagram for explaining an overview of a data processing system S. [Figure 2] FIG. 2 is a diagram showing a processing flow in the data processing device 1. [Figure 3] 1 is a diagram illustrating a configuration of a data processing device 1. FIG. [Figure 4] FIG. 10 is a diagram showing the relationship between partial text data and labels. [Figure 5] FIG. 10 is a diagram illustrating an example of hierarchically assigned labels. [Figure 6] FIG. 10 is a diagram illustrating an example of conversation structured data. [Figure 7] FIG. 10 is a diagram showing an example of structured conversation data corresponding to the evaluation item “stress tolerance.” [Figure 8] FIG. 10 is a diagram showing the relationship between the number of positive comments made during an interview and the rate at which applicants declined selection. [Figure 9] FIG. 10 is a diagram showing the relationship between the number of times the attractiveness of an organization's business or job is promoted and the rate at which applicants decline selection. [Figure 10] FIG. 10 is a diagram showing the relationship between the number of objective statements or the number of subjective statements made by an interviewer and the rate at which applicants decline selection. [Figure 11] 10 is a diagram showing an example of an evaluation result output by an evaluation unit 134. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0023] [Data Processing System S Overview] 1 is a diagram for explaining an overview of a data processing system S. The data processing system S is a system for evaluating at least one participant, an applicant A who is being interviewed for employment at an organization such as a company or a government office, and an interviewer B who is interviewing the applicant, based on conversation text data that records a conversation between the applicant A and the interviewer B who is interviewing the applicant.

[0024] The data processing system S includes a data processing device 1, multiple information terminals 2 (information terminal 2-1, information terminal 2-2), and an LLM device 3. The multiple information terminals 2 are computers used by applicant A and interviewer B. Information terminal 2-1 transmits voice data based on the voice uttered by applicant A via the network, and outputs the voice uttered by interviewer B from a speaker based on the voice data received via the network. Similarly, information terminal 2-2 transmits voice data based on the voice uttered by interviewer B via the network, and outputs the voice uttered by applicant A from a speaker based on the voice data received via the network.

[0025] The data processing device 1 is, for example, a server managed by an organization or a server managed by a business that provides a service for evaluating participants. In addition to the function of evaluating interview participants, the data processing device 1 may also provide a communication function for conducting online interviews. The data processing device 1 acquires conversation text data converted from voice data transmitted and received between multiple information terminals 2, and stores the conversation text data in a storage medium. The data processing device 1 evaluates the participants based on the conversation text data. In this embodiment, an example is shown in which the data processing device 1 acquires conversation text data from an external device that converts voice data into conversation text data, but the data processing device 1 may also convert voice data into conversation text data.

[0026] The LLM device 3 is a computer having a large-scale language model (LLM). The LLM device 3 has a model consisting of a neural network including a large number of data sets and a large number of parameters constructed using deep learning technology. The LLM device 3 performs natural language processing on input text data based on input instructions and outputs the text data after natural language processing. The LLM device 3 outputs, for example, a summary (summary sentence or summary words) of the input text data. The data processing device 1 may have the functions of the LLM device 3.

[0027] 2 is a diagram showing the flow of processing in the data processing device 1. When the data processing device 1 acquires conversation text data (S11), it divides the conversation text data into a plurality of partial text data by analyzing the conversation text data through natural language processing, for example, using the LLM device 3 (S12). Based on the results of summarizing each part, the data processing device 1 assigns labels to each partial text data indicating attributes such as the topic indicated by each partial text data or the characteristics of the participants (S13).

[0028] The data processing device 1 creates conversation structured data that is composed of a plurality of labels corresponding to a plurality of partial text data in the conversation text data and indicates the structure of the conversation (S14). The data processing device 1 creates conversation structured data corresponding to the participants or conversation structured data corresponding to the interviewer.

[0029] The data processing device 1 evaluates at least one of the participants or the interviewer based on the created conversation structured data. The data processing device 1 evaluates the interviewer, for example, by comparing the conversation structured data of the interviewer with reference structured data used for evaluating interviewers (S15). The reference structured data corresponds to an interview procedure that the organization considers desirable, and is, for example, conversation structured data of other interviewers who have a track record of having a low rate of applicant withdrawals.

[0030] Similarly, the data processing device 1 may evaluate an applicant by comparing the applicant's conversation structured data with reference structured data used to evaluate the applicant. In this case, the reference structured data corresponds to the speech patterns of applicants that the organization desires to hire, and is, for example, structured data based on conversation text data from interviews with people who have achieved good work performance after joining the organization. The reference structured data may be data created by modifying the conversation structured data of multiple interviewers or multiple applicants.

[0031] The data processing device 1 outputs the evaluation results (S16). For example, the data processing device 1 outputs the interviewer's evaluation results in association with the interviewer's name, and outputs the applicant's evaluation results in association with the applicant's name. The data processing device 1 transmits the evaluation results to, for example, an information terminal (e.g., a computer) in the human resources department. The information terminal transmits information for identifying the applicant or interviewer (e.g., name or identification number) to the data processing device 1, and thereby obtains the evaluation results of the applicant or interviewer corresponding to the transmitted information from the data processing device 1 and displays the obtained evaluation results.

[0032] 1 shows an online interview between applicant A and interviewer B, the interview is not limited to an online interview and may be an interview in which applicant A and interviewer B meet face-to-face. In other words, the conversation text data used by the data processing device 1 to evaluate participants may be conversation text data created based on audio recording data of a face-to-face interview.

[0033] In the past, in order to evaluate interview participants, evaluators had to watch recorded videos of the interviews. In order to evaluate participants in this way, evaluators had to watch the recorded videos for a long time. In addition, because evaluators' subjectivity is involved in evaluating participants, there was an issue that the evaluation content varied depending on the evaluator.

[0034] In response to this, the data processing device 1 creates conversation structured data based on the conversation text data of the participants during the interview as described above, and evaluates the participants based on the conversation structured data. This eliminates the need for the evaluator to spend a long time evaluating the participants, and also reduces the possibility of the evaluator's subjectivity being involved, thereby efficiently improving the quality of the evaluation. The configuration and operation of the data processing device 1 will be described in detail below.

[0035] [Configuration of data processing device 1] 3 is a diagram showing the configuration of the data processing device 1. The data processing device 1 has a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 has a data acquisition unit 131, a label assignment unit 132, a data creation unit 133, and an evaluation unit 134.

[0036] The communication unit 11 has a communication interface for transmitting and receiving data to and from other devices. For example, the communication unit 11 receives voice data transmitted and received by the information terminal 2 during an online interview, and inputs the received voice data to the data acquisition unit 131. The communication unit 11 also transmits conversation text data input from the label assignment unit 132 to the LLM device 3, receives summary text data from the LLM device 3, and inputs the received summary text data to the label assignment unit 132. The communication unit 11 may transmit the evaluation result output by the evaluation unit 134 to an external information terminal (e.g., a computer).

[0037] The storage unit 12 has storage media such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive). The storage unit 12 stores programs executed by the control unit 13. The storage unit 12 also stores conversation text data, summary text data, evaluation results, etc. in association with information for identifying participants (for example, names or IDs).

[0038] The storage unit 12 also stores data used by the evaluation unit 134 when evaluating participants. The storage unit 12 stores, for example, multiple pieces of standard structured data associated with multiple evaluation items for evaluating applicants or interviewers. The evaluation items correspond to, for example, one or more abilities (competencies) required of applicants or interviewers. Details of the standard structured data will be described later.

[0039] The control unit 13 has, for example, a CPU (Central Processing Unit). The control unit 13 executes the programs stored in the storage unit 12, thereby functioning as a data acquisition unit 131, a label assignment unit 132, a data creation unit 133, and an evaluation unit 134.

[0040] The data acquisition unit 131 acquires conversation text data indicating the content of the conversation between the applicant and the interviewer during the interview via the communication unit 11. The data acquisition unit 131 inputs the acquired conversation text data to the label assignment unit 132. The data acquisition unit 131 may input the conversation text data to the label assignment unit 132 via the storage unit 12 by storing the conversation text data in the storage unit 12 in association with the applicant and the interviewer.

[0041] The label assignment unit 132 performs natural language processing on the conversation text data to assign labels indicating attributes of the content of each partial text data to multiple partial text data included in the conversation text data. As an example, the label assignment unit 132 first transmits to the LLM device 3 the conversation text data corresponding to one interview, along with instruction text data that instructs the LLM device 3 to create summary text data that summarizes each utterance by the applicant and the interviewer included in the conversation text data. The label assignment unit 132 may transmit the partial text data corresponding to one utterance by the applicant and the interviewer to the LLM device 3 multiple times, and obtain the summary text data corresponding to each partial text data from the LLM device 3.

[0042] The label assignment unit 132 identifies attributes of the content of the partial text data based on words included in the summarized text data acquired from the LLM device 3. Attributes are represented by topics that are often discussed during interviews, such as self-introduction, icebreaker, academic background (what the applicant focused on during their student days), strengths and weaknesses, evaluations from others, leadership experience, and selection status. Attributes may also be characteristics of applicants, such as ambition, sense of responsibility, humility, and honesty. The label assignment unit 132 assigns labels to the partial text data by associating labels including text representing attributes with each of the multiple partial text data or multiple summarized text data and storing them in the storage unit 12.

[0043] FIG. 4 illustrates the relationship between partial text data and labels. The table in FIG. 4 associates the original text of partial text data corresponding to statements made by interviewers and applicants with their summaries, topic labels indicating the topic attributes of the statements, and feature labels indicating the characteristics of the applicants. The label assignment unit 132 determines the labels to assign based on, for example, the degree of match between words included in the summarized text data and multiple words associated with multiple pre-created topic labels or feature labels. Specifically, the label assignment unit 132 assigns to the summarized text data the topic label that most frequently matches multiple words included in the summarized text data among the pre-created topic labels or multiple words associated with the topic labels. The label assignment unit 132 may acquire the labels from the LLM device 3 along with the summarized text data.

[0044] The label assignment unit 132 may assign labels hierarchically. For example, the label assignment unit 132 may assign a label set as a label at a lower level than "School Background" to each of a plurality of partial text data to which the topic label "School Background" is assigned.

[0045] Fig. 5 is a diagram showing an example of hierarchically assigned labels. In Fig. 5, the second-level labels below the first-level label "scholarship" are "stress tolerance," "cooperativeness," and "ambition," which are assessment items confirmed during a conversation in which the interviewer confirms what the applicant focused on during their student days. Then, for example, the third-level labels below the "stress tolerance" label include labels such as "introduction," "specific confirmation," "process confirmation," and "emotion confirmation," which correspond to the statements made by the interviewer to confirm this assessment item.

[0046] The label assignment unit 132 assigns lower-level labels by, for example, selecting a label that best suits the content of the summarized text data from a plurality of labels predetermined as candidates for lower-level labels of higher-level labels. Alternatively, label candidates may not be predetermined, and the label assignment unit 132 may assign lower-level labels based on words contained in the summarized text data. By assigning labels at multiple levels in this way by the label assignment unit 132, the data creation unit 133 can create conversation structured data that represents the structure of various utterances by applicants and interviewers.

[0047] The data creation unit 133 creates conversation structured data composed of multiple labels corresponding to multiple partial text data in the conversation text data. The data creation unit 133 creates conversation structured data indicating the structure of the conversation, for example, by arranging the labels based on the order of the multiple partial text data. The data creation unit 133 creates conversation structured data, for example, by arranging the topic labels shown in FIG. 4 in chronological order of the utterances. When multiple identical topic labels appear consecutively, the data creation unit 133 creates conversation structured data by replacing the multiple topic labels with a single topic label. The number of times a single topic is spoken varies greatly depending on the interview, but by performing this replacement process by the data creation unit 133, the variation in conversation structured data between interviews is reduced, making it easier for the characteristics of the interviewer's interview process to be reflected in the conversation structured data.

[0048] Figure 6 is a diagram showing an example of conversation structured data. At the top of Figure 6, reference structured data is shown. Below that, conversation structured data for four interviews conducted by interviewer X and conversation structured data for four interviews conducted by interviewer Y are shown. "Self-introduction," "Icebreaker," "School Background," etc. are topic labels exemplified in Figure 4.

[0049] The data creating unit 133 may create conversation structured data in association with each of the multiple evaluation items. The data creating unit 133 creates conversation structured data corresponding to the evaluation items by arranging lower-level labels assigned to multiple partial text data to which evaluation item labels (e.g., "confirmation of stress tolerance") have been assigned.

[0050] FIG. 7 is a diagram showing an example of conversation structured data corresponding to the evaluation item "stress tolerance." The conversation structured data shown in FIG. 7 shows the flow of questions asked by the interviewer to check the applicant's stress tolerance. The multiple questions asked by the interviewer are each given a label: "introduction," "confirm specific examples," "confirm process," "confirm specific examples," and "confirm feelings." Associated with each label are also summarized text data of the interviewer's remarks (questions) and summarized text data of the applicant's remarks (answers).

[0051] The labels shown in FIG. 7 include a first label indicating an attribute that the question has an abstract content (hereinafter referred to as an "abstract question"), and a second label indicating an attribute that the question has a concrete content (hereinafter referred to as a "specific question"). The labels "Introduction," "Process Confirmation," and "Feeling Confirmation" are examples of first labels corresponding to abstract questions, and the label "Specific Example Confirmation" is an example of a second label corresponding to a specific question. In the frame of the summary text data of the interviewer's remarks, abstract questions are indicated by solid lines, and specific questions are indicated by dashed lines. The interviewer alternates between abstract questions and specific questions, with four transitions between abstract and specific questions.

[0052] The inventors have found that the more times there are transitions between abstract questions and specific questions, the easier it is to elicit answers that are useful in bringing out the applicant's abilities, personality, etc. Information that is included in the conversation structured data and indicates the number of transitions between abstract questions and specific questions is used by the evaluation unit 134, which will be described later, when evaluating the interviewer.

[0053] [Evaluation method] The evaluation unit 134 evaluates at least one of the applicant and the interviewer based on the pattern of multiple labels included in the conversation structured data. The pattern of multiple labels is, for example, a combination of the contents of the multiple labels, the order of the multiple labels, or the number of the multiple labels. As an example, the evaluation unit 134 evaluates at least one of the applicant and the interviewer by comparing reference structured data, which serves as a standard for evaluating a conversation and includes multiple labels, with the conversation structured data created by the data creation unit 133. The reference structured data may be structured data created in advance, or may be structured data of other applicants or interviewers who have already been evaluated.

[0054] For example, to evaluate interviewer X shown in Fig. 6, the evaluation unit 134 compares the reference structured data shown in Fig. 6 with a plurality of pieces of conversation structured data corresponding to a plurality of interviews conducted by interviewer X. The evaluation unit 134 evaluates at least one of the applicant and the interviewer based on the differences between the patterns of the plurality of labels in the conversation structured data and the patterns of the plurality of labels in the reference structured data.

[0055] For example, the evaluation unit 134 will give a higher evaluation to an interviewer the more labels included in the conversation structured data out of the multiple labels included in the reference structured data. Furthermore, the evaluation unit 134 will give a higher evaluation to an interviewer the more the order of the multiple labels included in both the conversation structured data and the reference structured data matches. In other words, the evaluation unit 134 will give a lower evaluation to an interviewer the greater the degree of difference between the label patterns in the conversation structured data and the label patterns in the reference structured data. By evaluating interviewers in this way, the evaluation unit 134 will give a higher evaluation to interviewers who are able to conduct interviews in accordance with the interview flow that the organization considers desirable, which will motivate interviewers to conduct interviews in a manner that conforms to the reference structured data.

[0056] The evaluation unit 134 may lower the evaluation of the interviewer if the number of labels not included in the reference structured data among the multiple labels included in the conversation structured data is equal to or greater than a threshold. The threshold is, for example, 30% of the number of labels included in the reference conversation structured data. By having the evaluation unit 134 evaluate the interviewer in this way, it is possible to prevent the interviewer from asking too many unnecessary questions.

[0057] The evaluation unit 134 may evaluate at least one of the applicant and the interviewer by comparing the conversation structured data associated with a target evaluation item (e.g., "stress tolerance") for evaluating the applicant or interviewer with the reference structured data associated with the target evaluation item, using the conversation structured data shown in FIG. 7, for example. As an example, if the label patterns in the reference structured data corresponding to the evaluation item "stress tolerance" are "introduction," "confirm specific examples," "confirm process," "confirm specific examples," and "confirm feelings," the label pattern of the conversation structured data shown in FIG. 7 matches the label pattern of the reference structured data. In such a case, the evaluation unit 134 increases the evaluation of the interviewer corresponding to the conversation structured data.

[0058] When the conversation structured data is "introduction," "process confirmation," and "emotion confirmation," and the label pattern of the conversation structured data does not match the label pattern of the reference structured data corresponding to the same evaluation item, the evaluation unit 134 lowers the interviewer's evaluation of the target evaluation item compared to when the label pattern matches. By the evaluation unit 134 evaluating the interviewer in this way, the interviewer can recognize evaluation items that he or she is weak at, and is motivated to ask questions in a flow appropriate to the evaluation items.

[0059] As described above, the inventors have found that the more transitions between abstract questions and specific questions there are, the more likely it is that useful answers will be elicited in bringing out the applicant's abilities, personality, etc. Therefore, the evaluation unit 134 may give a higher evaluation to an interviewer the more transitions there are between an interviewer's question corresponding to a label indicating an abstract question and an interviewer's question corresponding to a label indicating a specific question.

[0060] For example, the evaluation unit 134 evaluates interviewers whose number of transitions is equal to or greater than a threshold value higher than interviewers whose number of transitions is less than the threshold value. If the threshold value is two, the evaluation unit 134 will evaluate the interviewer corresponding to the conversation structured data shown in FIG. 7, in which transitions between abstract questions and specific questions occur four times, higher than average. By the evaluation unit 134 making such an evaluation, the interviewer will be conscious of switching between abstract questions and specific questions while conducting the interview, thereby improving the quality of the interview.

[0061] The inventors analyzed the relationship between the content of many interviews and the applicant's rejection rate, and found several interview trends that tend to lower the rejection rate. As one example, the inventors found a correlation between the number of times the interviewer made affirmative comments about the applicant and the applicant's rejection rate.

[0062] Figure 8 shows the relationship between the number of positive comments made during an interview and the rate at which applicants declined the selection process. Figure 8(a) shows the relationship between the number of all positive comments made and the rate at which applicants declined the selection process. Figure 8(b) shows the relationship between the number of positive comments made and the reasons given, and the rate at which applicants declined the selection process. Figure 8(c) shows the relationship between the number of positive comments made and the explanation of the relevance of the comments made by the applicant to the organization's business, and the rate at which applicants declined the selection process.

[0063] As shown in Figure 8(a), the more affirmative statements there are, the lower the decline rate. As shown in Figure 8(b), the more affirmative statements with reasons there are, the lower the decline rate. As shown in Figure 8(c), the more affirmative statements there are that include explanations of the relevance to the organization's work, the lower the decline rate.

[0064] Therefore, the evaluation unit 134 may evaluate at least one of the applicant and the interviewer based on the number of predetermined labels associated with a decrease in the decline rate, which are included in the conversation structured data. For example, the evaluation unit 134 may give a higher evaluation to the interviewer the greater the number of labels corresponding to the attribute that the interviewer affirms the applicant's statement. Examples of affirmative statements about the applicant's statement include "That's right," "That's good," and "I agree." It is believed that applicants are more likely to have a favorable impression of the interviewer if their own statements are affirmed, and therefore, by having the evaluation unit 134 evaluate the interviewer in this way, the interviewer will be more conscious of making affirmative statements about the applicant's statements.

[0065] Furthermore, the evaluation unit 134 may evaluate the interviewer higher the more labels there are corresponding to the attribute that the statement explains the reason why the interviewer affirms the applicant's statement. The evaluation unit 134 may evaluate the interviewer higher the more predetermined labels there are corresponding to the attribute that the statement explains the relevance of the applicant's statement to the work of the organization to which the interviewer belongs. By the evaluation unit 134 evaluating the interviewer in this way, the interviewer is motivated not only to simply affirm the applicant's statement but also to present the reasons and explain the relevance to the work.

[0066] The inventors also found a tendency that the more frequently an applicant appeals to the attractiveness of the organization's business or job, the lower the applicant's rejection rate. Figure 9 is a diagram showing the relationship between the number of times the attractiveness of the organization's business or job is appealed and the applicant's rejection rate. As shown in Figure 9, the more frequently an applicant appeals to the attractiveness of the organization's business or job, the lower the rejection rate.

[0067] Therefore, the evaluation unit 134 may evaluate the interviewer higher the more predetermined labels there are corresponding to the attribute that the statement is made by the interviewer explaining the attractiveness of the business or job at the organization to which the applicant is applying. By the evaluation unit 134 evaluating the interviewer in this way, the interviewer is motivated to explain the attractiveness of the business or job at the organization.

[0068] The inventors also found a tendency that the more subjective statements an interviewer makes, the lower the applicant rejection rate. Subjective statements include statements that use the interviewer as the subject, statements that include the interviewer's own experiences, or statements that include the interviewer's own opinions. Non-subjective statements (i.e., objective statements) include statements that simply present what is defined in the organization or statements that simply explain the actual situation of the organization.

[0069] FIG. 10 shows the relationship between the number of objective statements or the number of subjective statements made by an interviewer and the applicant's selection decline rate. FIG. 10(a) shows the relationship between the number of objective statements made by an interviewer and the decline rate. In FIG. 10(a), no clear trend is observed in the relationship between the number of objective statements made by an interviewer and the decline rate; rather, the more objective statements made by an interviewer, the higher the decline rate. FIG. 10(b) shows the relationship between the number of subjective statements made by an interviewer and the decline rate. In FIG. 10(b), there is a clear tendency that the more subjective statements made by an interviewer, the lower the decline rate. The inventors further found a tendency that the more specific the content of the interviewer's statements, the lower the applicant's decline rate.

[0070] Therefore, the evaluation unit 134 may evaluate the interviewer higher the more predetermined labels there are corresponding to the attribute that the statement indicates the subjective opinion of the interviewer. Furthermore, the evaluation unit 134 may evaluate the interviewer higher the more predetermined labels there are corresponding to the attribute that the statement includes a specific explanation. By the evaluation unit 134 evaluating the interviewer in this way, the interviewer is motivated to specifically state his or her subjective opinion.

[0071] The skill of an interviewer may affect the content of an applicant's answers. For example, a skilled interviewer may be able to extract more information from an applicant. Therefore, the evaluation unit 134 may evaluate an interviewer higher when multiple labels included in the conversation structured data corresponding to an applicant satisfy a predetermined condition than when the predetermined condition is not satisfied. The predetermined condition may be, for example, that the number of labels included in the conversation structured data is equal to or greater than a threshold, or that the number of labels with a predetermined content in the conversation structured data is equal to or greater than a threshold.

[0072] As an example, the evaluation unit 134 may evaluate an interviewer higher the more types or amount of information indicated by the multiple labels included in the conversation structured data corresponding to the applicant. The evaluation unit 134 may determine that the more labels included in the conversation structured data corresponding to the applicant, the greater the amount of information extracted from the applicant, and may evaluate the interviewer higher. By the evaluation unit 134 evaluating the interviewer in this way, the interviewer is motivated to extract as much information as possible from the applicant.

[0073] Furthermore, if specific explanations can be received from the applicant, the probability of correctly assessing the applicant's abilities and personality increases, and the probability of hiring an applicant who is a good fit for the organization increases. Therefore, when the conversation structured data corresponding to the applicant satisfies the interviewer's evaluation criteria regarding specificity, the evaluation unit 134 may give the interviewer a higher evaluation than when the conversation structured data does not satisfy the evaluation criteria.

[0074] Furthermore, because an applicant who can give a specific explanation can be considered to have excellent explanation skills, the evaluation unit 134 may evaluate an applicant higher when the conversation structured data corresponding to the applicant satisfies the applicant's evaluation criteria for specificity than when the evaluation criteria are not satisfied. By using the conversation structured data in this way, the evaluation unit 134 can evaluate the applicant without being influenced by the interviewer's subjectivity.

[0075] The evaluation unit 134 determines the evaluation value of the applicant for each of the multiple indicators based on the pattern of multiple labels corresponding to each of the multiple indicators for evaluating the applicant. The pattern of multiple labels may be the number of labels corresponding to each indicator (i.e., the number of statements corresponding to each indicator), or may be the order of the statements corresponding to the multiple labels.

[0076] FIG. 11 is a diagram illustrating an example of an evaluation result output by the evaluation unit 134. The evaluation result shown in FIG. 11 includes text indicating the interviewer's overall evaluation, the average performance of the interviewer being evaluated associated with multiple indicators, the average performance within the organization, and the evaluation level for each indicator. As indicated by the "appropriate speech ability" indicator in the "AI index," the results include the number of transitions between abstract and specific questions, the number of affirmative statements about the applicant, the number of affirmative statements with reasons for the affirmation, the number of times the attractiveness of the business was appealed to, the number of subjective and specific statements, and the average number of each within the organization. By outputting such evaluation results by the evaluation unit 134, the interviewer can understand which indicators need improvement, making it easier for the interviewer to improve their skills.

[0077] [Effects of Data Processing Device 1] As described above, the label assignment unit 132 assigns labels indicating the attributes of the content of each partial text data to each partial text data included in the conversation text data by performing natural language processing on the conversation text data based on the voice data uttered by the applicant and interviewer during the interview. The data creation unit 133 then creates conversation structured data that indicates the structure of the conversation and is composed of multiple labels corresponding to the multiple partial text data in the conversation text data, and the evaluation unit 134 evaluates at least one of the applicant and the interviewer based on the patterns of the multiple labels included in the conversation structured data.

[0078] With the data processing device 1 configured in this way, applicants or interviewers can be evaluated objectively based on standards set by the organization, without the need for an evaluator to listen to audio data from the interview for a long period of time, thereby improving the efficiency and quality of the evaluation of applicants or interviewers.

[0079] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments. [Explanation of symbols]

[0080] 1 Data Processing Device 2. Information terminal 3 LLM device 11 Communications Department 12 Storage section 13 Control Unit 131 Data Acquisition Unit 132 Label assignment unit 133 Data Creation Department 134 Evaluation Department

Claims

1. a data acquisition unit that acquires conversation text data indicating the content of the conversation during the interview between the applicant and the interviewer; a labeling unit that performs natural language processing on the conversation text data to assign labels indicating attributes of the contents of each of the partial text data to a plurality of partial text data included in the conversation text data; a data creation unit that creates conversation structured data that indicates a structure of the conversation and is configured by a plurality of the labels corresponding to a plurality of the partial text data in the conversation text data; an evaluation unit that evaluates at least one of the applicant and the interviewer based on a plurality of patterns of the labels included in the conversation structured data; A data processing device having:

2. the evaluation unit evaluates at least one of the applicant and the interviewer by comparing the conversation structured data with reference structured data that serves as a standard for evaluating the conversation and includes the plurality of labels; 2. The data processing device according to claim 1.

3. the evaluation unit evaluates at least one of the applicant and the interviewer based on the content of differences between the plurality of label patterns in the conversation structured data and the plurality of label patterns in the reference structured data.

3. The data processing device according to claim 2.

4. a storage unit that stores a plurality of the reference structured data associated with a plurality of evaluation items that are used to evaluate the applicant or the interviewer; the data creation unit creates the conversation structured data in association with each of the plurality of evaluation items; the evaluation unit evaluates at least one of the applicant and the interviewer by comparing the conversation structured data associated with the target evaluation item for evaluating the applicant or the interviewer with the reference structured data associated with the target evaluation item; 3. The data processing device according to claim 2.

5. The labels include a first label indicating an attribute that the question has an abstract content and a second label indicating an attribute that the question has a specific content, the evaluation unit increases the evaluation of the interviewer as the number of transitions between the question of the interviewer corresponding to the first label and the question of the interviewer corresponding to the second label increases; 2. The data processing device according to claim 1.

6. the evaluation unit evaluates at least one of the applicant and the interviewer based on the number of predetermined labels included in the conversation structured data.

2. The data processing device according to claim 1.

7. the evaluation unit increases the evaluation of the interviewer as the number of the predetermined labels corresponding to the attribute that the statement of the applicant is a statement affirmed by the interviewer increases; 7. The data processing device according to claim 6.

8. the evaluation unit increases the evaluation of the interviewer as the number of the predetermined labels corresponding to the attribute that the statement is an explanation of the reason why the interviewer affirms the statement of the applicant increases; 7. The data processing device according to claim 6.

9. the evaluation unit increases the evaluation of the interviewer as the number of the predetermined labels corresponding to the attribute that the statement explains the relevance between the statement of the applicant and the work of the organization to which the interviewer belongs increases; 7. The data processing device according to claim 6.

10. the evaluation unit increases the evaluation of the interviewer as the number of the predetermined labels corresponding to the attribute that the statement is a statement by the interviewer explaining the appeal of a business or a position in the organization to which the applicant is applying increases; 7. The data processing device according to claim 6.

11. the evaluation unit increases the evaluation of the interviewer as the number of the predetermined labels corresponding to the attribute that the statement is an expression of the interviewer's subjective opinion increases, 7. The data processing device according to claim 6.

12. the evaluation unit increases the evaluation of the interviewer as the number of the predetermined labels corresponding to the attribute that the statement includes a specific explanation increases; 7. The data processing device according to claim 6.

13. the evaluation unit, when the plurality of labels included in the conversation structured data corresponding to the applicant satisfy a predetermined condition, evaluates the interviewer higher than when the predetermined condition is not satisfied; 2. The data processing device according to claim 1.

14. the evaluation unit, when the conversation structured data corresponding to the applicant satisfies an evaluation criterion regarding specificity, evaluates the applicant higher than when the conversation structured data does not satisfy the evaluation criterion; 2. The data processing device according to claim 1.

15. the evaluation unit determines an evaluation value of the applicant for each of a plurality of indicators for evaluating the applicant based on a plurality of label patterns corresponding to each of the plurality of indicators; 2. The data processing device according to claim 1.

16. The computer executes A step of acquiring conversation text data indicating the content of the conversation during the interview between the applicant and the interviewer; a step of assigning labels indicating attributes of the content of each of a plurality of partial text data included in the conversation text data by performing natural language processing on the conversation text data; creating conversation structured data that indicates a structure of the conversation and is configured by a plurality of the labels corresponding to a plurality of the partial text data in the conversation text data; evaluating at least one of the applicant and the interviewer based on patterns of the plurality of labels included in the conversation structured data; A data processing method comprising:

17. On the computer, A step of acquiring conversation text data indicating the content of the conversation during the interview between the applicant and the interviewer; a step of assigning labels indicating attributes of the content of each of a plurality of partial text data included in the conversation text data by performing natural language processing on the conversation text data; creating conversation structured data that indicates a structure of the conversation and is configured by a plurality of the labels corresponding to a plurality of the partial text data in the conversation text data; evaluating at least one of the applicant and the interviewer based on patterns of the plurality of labels included in the conversation structured data; A program to execute.

18. A data processing device and an information terminal are provided, The data processing device includes: a data acquisition unit that acquires conversation text data indicating the content of the conversation during the interview between the applicant and the interviewer; a labeling unit that performs natural language processing on the conversation text data to assign labels indicating attributes of the contents of each of the partial text data to a plurality of partial text data included in the conversation text data; a data creation unit that creates conversation structured data that indicates a structure of the conversation and is configured by a plurality of the labels corresponding to a plurality of the partial text data in the conversation text data; an evaluation unit that evaluates at least one of the applicant and the interviewer based on a plurality of patterns of the labels included in the conversation structured data; and the information terminal transmits information for identifying the applicant or the interviewer to the data processing device, and thereby obtains from the data processing device the results of the evaluation of the applicant or the interviewer corresponding to the transmitted information; Data processing system.

Images