A training method for a follow-up model and a follow-up method for interview questions

Abstract

The present invention relates to a method and device for training a follow-up question model, a method and device for following up interview questions, and two corresponding computer-readable storage media. The method includes the steps of: constructing a follow-up question bank according to at least one follow-up dimension of each source question in the interview question bank; performing follow-up result annotation on the answer integrity and answer depth of multiple answer samples according to each of the follow-up dimensions of each source question; constructing a follow-up question model; and training the follow-up question model to judge the follow-up dimension that needs to be followed up according to the answer content of the interview candidate based on the multiple answer samples and their corresponding follow-up result annotations. The present invention can automatically select corresponding follow-up questions according to the integrity and depth of the candidate's answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

Description

Technical Field

[0001] The present invention belongs to the technical field of video interviews, and particularly relates to a method for training a follow-up question model and a method for asking follow-up questions for interview questions. Background Art

[0002] A video interview refers to the act of an interviewer and a job candidate communicating through a camera, headset, and keyboard for voice, video, and text communication, so as to conduct a recruitment interview using the Internet and a computer. By adopting the method of video interviews, both parties can complete the recruitment interview process without leaving their homes, thereby breaking the spatial limitation to improve the recruitment efficiency.

[0003] Existing video interview methods include real-time video interviews and asynchronous video interviews. A real-time video interview requires the interviewer and the candidate to be online at the same time, which has great limitations on the interview time of both parties. Relatively speaking, in an asynchronous video interview, the interviewer can preset interview questions in the recruitment system. The candidate can answer these preset interview questions at any time, and the computer background automatically records and uploads their answers for the interviewer to view, evaluate, and share at any time and any place later. Therefore, asynchronous video interviews completely break the constraints of space and time, providing greater freedom for both parties in the interview.

[0004] However, the existing asynchronous video interview technology has the problem of lack of flexibility. When the candidate's answer is not complete enough, the interviewer may not be able to accurately judge the matching degree between the candidate and the recruitment position due to the lack of relevant information. In this case, even if the candidate participates in an asynchronous video interview, the interviewer still needs to communicate with the candidate again in real time, and by asking follow-up questions about the candidate's incomplete or ambiguous answers, can more comprehensively evaluate the matching degree between the candidate and the recruitment position. This defect seriously reduces the advantages of asynchronous video interviews and is not conducive to the development and popularization of asynchronous video interview technology.

[0005] Therefore, there is an urgent need in this field for an interview question follow-up technology to automatically select corresponding follow-up questions according to the completeness and depth of the candidate's answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position. Summary of the Invention

[0006] The following gives a brief overview of one or more aspects to provide a basic understanding of these aspects. This overview is not an exhaustive survey of all contemplated aspects, and is neither intended to identify key or decisive elements of all aspects nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to a more detailed description to follow.

[0007] To overcome the above-mentioned defects existing in the prior art, the present invention provides a training method for a follow-up question model, a follow-up question method for interview questions, a training device for a follow-up question model, a follow-up question device for interview questions, and two corresponding computer-readable storage media, which are used to automatically select corresponding follow-up questions according to the completeness and depth of the candidate's answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0008] The training method for the above-mentioned follow-up question model provided by the present invention includes the steps of: constructing a follow-up question bank according to at least one follow-up dimension of each source question in the interview question bank, wherein the interview question bank includes multiple source questions, and each follow-up dimension corresponds to at least one follow-up question in the follow-up question bank; marking the follow-up results of the answer completeness and answer depth of multiple answer samples according to each follow-up dimension of each source question, and the follow-up result marking indicates whether each follow-up dimension needs to be followed up; constructing a follow-up question model, which is suitable for judging the follow-up dimension that needs to be followed up according to the trained follow-up rules, and retrieving the corresponding follow-up question from the follow-up question bank according to the follow-up dimension that needs to be followed up; and training the follow-up question model to judge the follow-up dimension that needs to be followed up according to the answer content of the interview candidate according to the multiple answer samples and their corresponding follow-up result markings. By marking the dimensions that need to be followed up for the answer samples according to multiple follow-up dimensions of the source questions, the follow-up question model trained by the present invention can provide corresponding follow-up questions to supplement the deficiencies of the answer content according to the completeness and depth of the candidate's answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0009] The method for following up on the above-mentioned interview questions provided by the present invention includes the steps of: collecting the answers of the interview candidates to the interview questions, where the interview questions are source questions stored in the interview question bank, the interview question bank includes multiple such source questions, each source question includes multiple follow-up dimensions, and each follow-up dimension corresponds to at least one follow-up question in the follow-up question bank; inputting the answer content into a pre-trained follow-up model to use the follow-up model to judge the follow-up dimensions that need to be followed up, where the follow-up model judges the answer integrity and answer depth of the answer content according to each follow-up dimension of the interview question, and judges whether each follow-up dimension needs to be followed up according to the answer integrity and the answer depth; and using the follow-up model to retrieve the corresponding follow-up questions from the follow-up question bank according to the follow-up dimensions that need to be followed up for follow-up. By judging the integrity and depth of the candidate's answer content according to multiple follow-up dimensions of the interview question, the follow-up model adopted by the present invention can provide corresponding follow-up questions for the deficiencies in the answer content to help the candidate supplement the answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0010] The training device of the above-mentioned follow-up model provided by the present invention includes a memory and a processor. The processor is connected to the memory and is configured to implement a training method of a follow-up model. By marking the follow-up dimensions that need to be followed up for the answer samples according to multiple follow-up dimensions of the source questions, the follow-up model trained by the present invention can provide corresponding follow-up questions according to the integrity and depth of the candidate's answer content to supplement the deficiencies in the answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0011] The follow-up device for the above-mentioned interview questions provided by the present invention includes a memory and a processor. The processor is connected to the memory and is configured to implement a method for following up on interview questions. By judging the integrity and depth of the candidate's answer content according to multiple follow-up dimensions of the interview question, the follow-up model adopted by the present invention can provide corresponding follow-up questions for the deficiencies in the answer content to help the candidate supplement the answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0012] The first computer-readable storage medium provided by the present invention stores computer instructions thereon. When the computer instructions are executed by a processor, a training method for a follow-up question model can be implemented. By annotating the integrity and depth of answer samples according to multiple follow-up question dimensions of a source question, the follow-up question model trained by the present invention can judge the integrity and depth of the answer content of a candidate, and provide corresponding follow-up questions to supplement the deficiencies in the answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0013] The second computer-readable storage medium provided by the present invention stores computer instructions thereon. When the computer instructions are executed by a processor, a follow-up question method for interview questions can be implemented. By judging the integrity and depth of the answer content of a candidate according to multiple follow-up question dimensions of an interview question, the follow-up question model adopted by the present invention can provide corresponding follow-up questions for the deficiencies in the answer content to help the candidate supplement the answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] After reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings, the above features and advantages of the present invention can be better understood. In the drawings, the components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0015] Figure 1 FIG. shows a schematic flowchart of a training method for a follow-up question model provided according to an aspect of the present invention.

[0016] Figure 2 FIG. shows a schematic architecture diagram of a training device for a follow-up question model provided according to some embodiments of the present invention.

[0017] Figure 3 FIG. shows a schematic architecture diagram of a source question, a follow-up question dimension, and a follow-up question provided according to some embodiments of the present invention.

[0018] Figure 4 FIG. shows a schematic diagram of a source question entry provided according to some embodiments of the present invention.

[0019] Figure 5 FIG. shows a schematic architecture diagram of a follow-up question device for interview questions provided according to some embodiments of the present invention.

[0020] Figure 6 FIG. shows a schematic flowchart of a follow-up question method for interview questions provided according to an aspect of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The following specific embodiments illustrate the implementation manners of the present invention, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention will be introduced in conjunction with preferred embodiments, this does not mean that the features of this invention are limited to this implementation manner. On the contrary, the purpose of introducing the invention in conjunction with the implementation manner is to cover other alternatives or modifications that may be extended based on the claims of the present invention. In order to provide a deep understanding of the present invention, many specific details will be included in the following description. The present invention can also be implemented without these details. In addition, in order to avoid confusing or obscuring the focus of the present invention, some specific details will be omitted in the description.

[0022] It can be understood that although terms such as "first", "second", "third", etc. can be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first component, region, layer, and / or part discussed below can be referred to as the second component, region, layer, and / or part without departing from some embodiments of the present invention.

[0023] As described above, the existing asynchronous video interview technology has the problem of lack of flexibility. When the candidate's answer is not complete, the interviewer may not be able to accurately judge the matching degree between the candidate and the recruitment position due to the lack of relevant information. In this case, even if the candidate participates in an asynchronous video interview, the interviewer still needs to communicate with the candidate again in person, and by asking questions about the candidate's incomplete or ambiguous answers, can more comprehensively evaluate the matching degree between the candidate and the recruitment position. This defect seriously reduces the advantages of asynchronous video interviews and is not conducive to the development and popularization of asynchronous video interview technology.

[0024] In order to overcome the above-mentioned defects existing in the prior art, the present invention provides a training method for a follow-up question model, a method for asking follow-up questions for interview questions, a training device for a follow-up question model, a device for asking follow-up questions for interview questions, and two corresponding computer-readable storage media, which are used to automatically select corresponding follow-up questions according to the completeness and depth of the candidate's answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0025] Please refer to Figure 1 , Figure 1 which shows a schematic flowchart of a training method for a follow-up question model provided according to an aspect of the present invention.

[0026] As Figure 1 shown, the above-mentioned training method for the follow-up question model provided by the present invention may include step 101: constructing a follow-up question bank according to at least one follow-up dimension of each source question in the interview question bank.

[0027] The above interview question bank stores multiple source questions preset by the interviewer in advance, which are used to guide the interviewee candidates to answer the information that the interviewer hopes to understand, so as to help the interviewer judge the matching degree between the candidate and the recruitment position. Compared with the interview questions used in the existing asynchronous video interview technology, the source questions adopted by the present invention include at least one follow-up dimension for assessing at least one competency of the candidate. Each follow-up dimension can correspond to one competency, so that the interviewer can assess multiple competencies of the candidate through one source question at the same time.

[0028] In some embodiments, the source questions stored in the interview question bank can be divided into two categories. The first category of source questions mainly assess the comprehensive quality of the candidate according to the completeness of the candidate's answer content. The first category of source questions can include multiple first follow-up dimensions. Each first follow-up dimension indicates a general competency of the candidate. The general competency includes, but is not limited to, soft skills such as communication ability, stress resistance ability, and teamwork ability, which are applicable to almost all positions. The interviewer can set multiple first follow-up dimensions for each first category of source questions to comprehensively understand multiple general competencies of the candidate.

[0029] Relatively, the second category of source questions mainly assess the professional quality of the candidate according to the depth of the candidate's answer content. The second category of source questions can include multiple second follow-up dimensions. Each second follow-up dimension indicates a position competency of the candidate. The position competency includes, but is not limited to, financial knowledge for financial positions, human resources knowledge for personnel positions, and programming technology for programming positions. Each position competency can indicate a knowledge point to be assessed by the corresponding second category of source questions. The interviewer can judge whether the candidate comprehensively understands each knowledge point of the corresponding professional skill according to whether the candidate's answer involves each second follow-up dimension. Further, the interviewer can set multiple assessment dimensions for each second follow-up dimension and further judge the depth of the candidate's mastery of the knowledge point according to whether the candidate's answer involves each assessment dimension. Furthermore, each second follow-up dimension can include multiple levels of assessment dimensions. Specifically, each second follow-up dimension can include multiple first-level assessment dimensions. Each first-level assessment dimension can include multiple second-level assessment dimensions. Each second-level assessment dimension can include multiple third-level assessment dimensions. And so on, expanding the in-depth follow-up of professional quality from point to surface. The interviewer can accurately judge the specific depth of the candidate's mastery of the knowledge point according to which level of assessment dimension the candidate's answer finally involves.

[0030] Those skilled in the art can understand that the above solution of using the first type of source questions to assess the comprehensiveness of the candidate's comprehensive qualities and using the second type of source questions to assess the depth of the candidate's professional qualities is only a non-limiting embodiment provided by the present invention, aiming to demonstrate the main concept of the present invention and provide a specific solution convenient for the public to implement, rather than limiting the protection scope of the present invention. Optionally, in some other embodiments, the present invention can also use the above first type of source questions to conduct integrity inquiries about the candidate's professional qualities, or use the above second type of source questions to conduct in-depth inquiries about the depth of the candidate's comprehensive qualities.

[0031] In some non-limiting embodiments, the above training method of the inquiry model provided by the present invention can be automatically implemented by a training device of the inquiry model. Please refer to Figure 2 , Figure 2 which shows a schematic architecture diagram of a training device of an inquiry model provided according to some embodiments of the present invention.

[0032] As Figure 2 shown, the above training device 20 of the inquiry model may include a memory 21 and a processor 22. The memory 21 may include a computer-readable storage medium, on which computer instructions are stored. The processor 22 is connected to the memory 21 and is configured to execute the computer instructions stored on the memory 21 to implement a training method of an inquiry model.

[0033] Specifically, when constructing the inquiry question bank, the processor 22 can configure at least one inquiry question for each inquiry dimension of each source question, for use in inquiring about the inquiry dimension where the answer integrity or answer depth does not meet the standard. Please refer to Figure 3 , Figure 3 which shows a schematic architecture diagram of source questions, inquiry dimensions and inquiry questions provided according to some embodiments of the present invention.

[0034] As Figure 3 shown, in some embodiments of the present invention, the interview question bank may include multiple first type of source questions and multiple second type of source questions, for use in assessing the comprehensive qualities and professional qualities of candidates respectively. In some embodiments, one first type of source question may include K first inquiry dimensions, for use in evaluating the integrity of the candidate's answer content. The processor 22 can configure S inquiry questions for each inquiry dimension, for the inquiry model to randomly select one inquiry question from them to conduct integrity inquiries about the candidate. At this time, this first type of source question can correspond to K*S inquiry questions. In some embodiments, the processor 22 can configure more than 20 inquiry questions (i.e., S>20) for each first inquiry dimension, to prevent candidates from deliberately preparing for the same inquiry questions using the experience of multiple interviews to cover up their true situation.

[0035] For example, the first type of source question 11 is "Please share the most difficult task you encountered in the past year. Specifically, what was the content of the task at that time? What difficulties did you encounter? How did you overcome them? What was the final result of the task?", including multiple first follow-up dimensions 111-114 such as background (Situation, S), content (Task, T), action (Action, A), and result (Result, R). For the first follow-up dimension 114 of the task result (R), the processor 22 can configure follow-up questions 1141 such as "What was the result of the final task?" for the follow-up model to select.

[0036] Similarly, a second type of source question can include K1 second follow-up dimensions for evaluating the integrity of the candidate's answer content. Each second follow-up dimension can further include K2 assessment dimensions for evaluating the depth of the candidate's answer content in that second follow-up dimension. The processor 22 can configure S follow-up questions for each assessment dimension for the follow-up model to randomly select one follow-up question to deeply question the candidate. At this time, define K = K1 * K2, then this second type of source question can also correspond to K * S follow-up questions. Similarly, the processor 22 can also configure more than twenty follow-up questions for each second follow-up dimension (i.e., S > 20) to prevent candidates from deliberately preparing for the same follow-up questions using their experience in multiple interviews to cover up their true situation.

[0037] For example, the second type of source question 21 for the Java engineer position is "Please introduce in detail the core components of the Java microservices architecture Spring Cloud", including multiple second follow-up dimensions 211-215 such as the Eureka component, Ribbon component, Feign component, and Zuul component. For the second follow-up dimension 211 of the Eureka component, it can further include multiple assessment dimensions 2111-2112 such as component function and usage method. For the assessment dimension 2111 of the component function, the processor 22 can configure follow-up questions 21111 such as "What is the main function of the Eureka component?" for the follow-up model to select.

[0038] In some embodiments, the processor 22 can construct a follow-up question library in the dictionary form of <source question, K * S follow-up questions>. The follow-up question library dictionary constructs source question entries in units of source questions. By including multiple follow-up dimensions of each source question and multiple follow-up questions of each follow-up dimension in the corresponding source question entry, the K * S follow-up questions can be associated with the corresponding source questions.

[0039] Please further refer to Figure 4 , Figure 4 shows a schematic diagram of a source question entry provided according to some embodiments of the present invention.

[0040] As Figure 4 shown, the entries of the first type of source question 11 may include four first follow-up dimensions 111 to 114. Each follow-up dimension (e.g., the follow-up dimension 111) may correspond to three follow-up questions 1111 to 1113. The processor 22 may construct the entries of the first type of source question 11 in the form of <the first type of source question 11, 4 * 3 follow-up questions>, and construct a follow-up question library dictionary by collecting the entries of all source questions. During an asynchronous video interview, the follow-up model may query the corresponding follow-up dimensions 111 to 114 according to the first type of source question 11 raised in the interview, and randomly select a follow-up question 1111 from the follow-up question library according to the first follow-up dimension 111 that needs to be followed up, and conduct an integrity follow-up on the candidate.

[0041] As Figure 1 shown, the training method of the above-mentioned follow-up model provided by the present invention may further include step 102: annotating the follow-up results of the answer integrity and answer depth of multiple answer samples according to each follow-up dimension of each source question.

[0042] The above-mentioned follow-up model provided by the present invention is an artificial intelligence (AI) model, and it is necessary to judge the follow-up dimension that needs to be followed up according to the pre-trained follow-up rules. In some embodiments, the trainer of the follow-up model may collect a large number of interview video samples involving different positions, and invite senior interviewers to act as annotators to conduct follow-up result annotation for the integrity and depth of each answer sample, so as to determine which follow-up dimensions need to be followed up for these answer samples.

[0043] As described above, in some embodiments, the source questions stored in the interview question bank may be divided into two categories. The first type of source question includes multiple first follow-up dimensions for evaluating the integrity of the answer content. The second type of source question includes multiple second follow-up dimensions for evaluating the depth of the answer content. Correspondingly, the annotator may also use different criteria to conduct annotation for the above two types of source questions.

[0044] Specifically, if the interview question is the above-mentioned first type of source question, the annotator may annotate the missing first follow-up dimension of the answer sample according to the richness of each first follow-up dimension of the answer sample for the first type of source question. The richness may be evaluated from aspects such as whether the answer content only repeats the question, whether the answer content is too broad, and whether the answer is too concise. If the content of the answer sample does not cover a certain first follow-up dimension, or the richness for a certain follow-up dimension does not meet the standard, the annotator may mark the missing first follow-up dimension as the follow-up dimension that needs to be followed up to guide the candidate to improve their answer content.

[0045] If the interview question is the second type of source question mentioned above, the annotator can, based on the answer samples, annotate the second follow-up dimensions with insufficient answer depth for the second type of source question. The answer depth can be evaluated according to the assessment dimensions involved in the answer content. If the content of the answer sample involves a certain second follow-up dimension but does not involve one of the assessment dimensions therein, the annotator can determine that the candidate knows the knowledge point but the answer depth is insufficient, and thus annotate the second follow-up dimension with insufficient answer depth as the follow-up dimension that needs to be pursued to guide the candidate to deepen the answer content.

[0046] Furthermore, for the embodiments in which the above-mentioned second follow-up dimension includes multiple levels of assessment dimensions, if the content of the answer sample involves a certain second follow-up dimension but does not involve one or more of the first-level assessment dimensions therein, the annotator can determine that the candidate knows the knowledge point corresponding to the second follow-up dimension but the answer depth is insufficient. Therefore, the annotator annotates the second follow-up dimension with insufficient answer depth as the follow-up dimension that needs to be pursued, and annotates the missing first-level assessment dimensions for the follow-up model to select follow-up questions. Thereafter, if the supplementary answer content of the answer sample involves a certain second-level assessment dimension of the missing first-level assessment dimension but does not involve one or more of the third-level assessment dimensions therein, the annotator can determine that the candidate knows the knowledge point corresponding to the second-level assessment dimension but the answer depth is still insufficient. Therefore, the annotator annotates the second-level assessment dimension with insufficient answer depth as the assessment dimension that needs to be pursued, and annotates the missing third-level assessment dimensions for the follow-up model to further select follow-up questions. And so on, the annotator can successively annotate the assessment dimensions that need to be pursued until the second type of source question no longer involves deeper-level assessment dimensions, or the answer sample no longer involves any lower-level dimensions of the follow-up questions.

[0047] In some preferred embodiments, in order to ensure the objectivity and accuracy of the annotated data, the trainer of the follow-up model can invite multiple senior interviewers to act as annotators together to annotate the same interview video sample to determine the follow-up dimensions that need to be pursued. Further, the number of the multiple annotators can be an odd number, so as to obtain a differential standard by voting when there are inconsistent annotation opinions.

[0048] As Figure 1 shown, the above-mentioned training method of the follow-up model provided by the present invention may further include step 102: constructing a follow-up model.

[0049] As described above, the follow-up model is an artificial intelligence (AI) model that determines the follow-up dimensions that need to be pursued according to pre-trained follow-up rules and retrieves the corresponding follow-up questions from the follow-up question library. In some embodiments, the follow-up model can be implemented based on a follow-up device for interview questions. Please refer to Figure 5 , Figure 5The figure shows a schematic architecture diagram of a follow-up question device for interview questions provided according to some embodiments of the present invention.

[0050] As Figure 5 shown, the above-mentioned follow-up question device 50 for interview questions may include a memory 51 and a processor 52. The memory 51 may include a computer-readable storage medium, on which computer instructions are stored. The processor 52 is connected to the memory 51 and is configured to execute the computer instructions stored on the memory 51 to implement a method for following up interview questions, so as to realize the functions of the above-mentioned follow-up model.

[0051] In some embodiments, the trainer of the follow-up model may configure the above-mentioned interview question bank and the above-mentioned follow-up question bank in the memory 51 for the follow-up model to query the corresponding follow-up dimension according to the source question involved in the interview, and retrieve the corresponding follow-up question from the follow-up question bank according to the follow-up dimension that needs to be followed up.

[0052] As Figure 1 shown, the above-mentioned training method for the follow-up model provided by the present invention may further include step 104: training the follow-up model to judge the follow-up dimension that needs to be followed up according to the answer content of the interview candidate according to multiple answer samples and their corresponding follow-up result annotations.

[0053] After obtaining the annotation data of the annotator for multiple answer samples, the processor 22 may use the multiple answer samples as input parameters of the follow-up model, and use the corresponding follow-up result annotations as output parameters of the follow-up model to train the follow-up model to judge the follow-up dimension that needs to be followed up according to the answer content.

[0054] In some embodiments, the follow-up model may include a classification module for judging whether follow-up is needed. The classification module may select a text classification model based on a deep neural network (CNN, RNN (LSTM, GRU), Bi-LSTM, Bi-GRU, etc.) or a text classification model based on a pre-trained model (GPT, BERT, ELMO and their variants).

[0055] When training the follow-up rule on whether follow-up questions are needed, the processor 52 can first perform speech recognition on each answer sample to obtain the answer text therein. After that, the processor 52 can respectively form multiple question-answer text pairs <Q, A> by combining the answer text A of each answer sample with the question text Q of the corresponding source question. At the same time, the processor 22 can determine whether follow-up questions are needed according to whether the follow-up result annotation indicates the follow-up dimension that requires follow-up questions. Specifically, if the follow-up result annotation indicates one or more dimensions that require follow-up questions, the processor 22 can label the binary classification output label of the classification module as 1 to indicate that follow-up questions are needed. On the contrary, if the follow-up result annotation does not indicate any dimension that requires follow-up questions, the processor 22 can label the binary classification output label of the classification module as 0 to indicate that follow-up questions are not needed.

[0056] After that, the processor 22 can use multiple question-answer text pairs <Q, A> as input and the output label indicating whether follow-up questions are needed as output to train the classification module to determine whether follow-up questions are needed based on the answer content of the interview candidate. The trained classification module can output the corresponding label 1 or 0 according to the input question-answer text pair <Q, A> to indicate whether follow-up questions are needed.

[0057] Furthermore, the follow-up model can also include a selection module for selecting the follow-up dimensions that require follow-up questions. As mentioned above, since this selection function involves a relatively high semantic understanding requirement for the question-answer content, the processor 52 needs to first input the answer text A and the question text Q of the corresponding source question into the encoder respectively to obtain the corresponding answer semantic representation vector repA and question semantic representation vector repQ. The encoder can be built based on one of a convolutional neural network (CNN), a recurrent neural network (RNN, LSTM, GRU, Bi-LSTM, Bi-GRU, etc.) or a pre-trained model (ELMO, GPT, BERT, etc.).

[0058] After that, the processor 22 can use the response semantic representation vectors repA of multiple response samples and their corresponding question semantic representation vectors repQ as inputs, and use the follow-up dimension to be followed up indicated by the corresponding follow-up result annotation as the output. The training selection module determines the dimension to be followed up based on the response content of the interview candidate. For example, for the four STAR follow-up dimensions 111-114 of the above-mentioned first type of source question 11, the processor 22 can use a set of label sets (C0, C1, C2, C3) with a dimension of 4 to indicate the dimension to be followed up. Specifically, C0 indicates whether the background (S) dimension 111 needs to be followed up. If the S dimension 111 does not need to be followed up, the selection module can output C0 = 0. Conversely, if the S dimension 111 needs to be followed up, the selection module can output C0 = 1. Similarly, C1, C2, and C3 can respectively indicate whether the first follow-up dimensions 112-114 corresponding to TAR need to be followed up. The trained selection module can output the corresponding label set (C0, C1, C2, C3) according to the input response semantic representation vector repA and its corresponding question semantic representation vector repQ to indicate the dimension to be followed up.

[0059] In some more optimal embodiments, in order to further improve the semantic understanding ability of the selection module for the Q&A content, the processor 52 can perform further Q&A interaction on the obtained response semantic representation vector repA and question semantic representation vector repQ to obtain the corresponding Q&A correlation representation vector repQA. After that, the processor 52 can perform weighted fusion on the response semantic representation vector repA, the corresponding question semantic representation vector repQ, and the corresponding Q&A correlation representation vector repQA of each response sample respectively to obtain the final correlation representation vector repFinal corresponding to each response sample, so as to map repQ, repA, and repQA to the same high-dimensional space for better classification.

[0060] After that, the processor 22 can use the final correlation representation vectors repFinal of multiple response samples as inputs, and use the corresponding follow-up result annotations (C0, C1, C2, C3) as outputs. The training selection module determines the dimension to be followed up based on the response content of the interview candidate. The trained selection module can output the corresponding label set (C0, C1, C2, C3) according to the input final correlation representation vector repFinal to more accurately indicate the dimension to be followed up.

[0061] Based on the above description, by annotating the dimensions that need to be followed up for the answer samples according to multiple follow-up dimensions of the source question, the follow-up model trained by the present invention can provide corresponding follow-up questions according to the completeness and depth of the candidate's answer content to supplement the deficiencies of the answer content. By applying the trained follow-up model to asynchronous video interviews, it can ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0062] According to another aspect of the present invention, the present document also provides a method for following up interview questions. Please refer to Figure 6 , Figure 6 which shows a schematic flowchart of the interview question follow-up method provided according to one aspect of the present invention.

[0063] As Figure 6 shown, the above-mentioned interview question follow-up method provided by the present invention may include step 601: collecting the answer content of the interview candidate to the interview question.

[0064] The above-mentioned interview questions may be source questions stored in the interview question bank. As described above, the interview question bank may include multiple source questions for guiding the interview candidate to answer the information that the interviewer hopes to understand, so as to facilitate the interviewer to judge the matching degree between the candidate and the recruitment position. Compared with the interview questions adopted by the existing asynchronous video interview technology, the source questions adopted by the present invention may include at least one follow-up dimension for assessing at least one competency of the candidate. Each follow-up dimension may correspond to one competency, so that the interviewer can assess multiple competencies of the candidate through one source question at the same time.

[0065] In some embodiments, the above-mentioned interview question follow-up method provided by the present invention may be applied to asynchronous video interviews. In some non-limiting embodiments, the interview question follow-up method may be implemented based on Figure 5 the follow-up device 50 shown. Specifically, during the asynchronous video interview process, the candidate can use the computer's camera and microphone to collect their interview video data and upload the collected interview video data to the system server of the asynchronous video interview in real time through the network. The processor 52 of the follow-up device 50 can obtain the interview video data uploaded by the candidate in real time and obtain the answer content therein through methods such as speech recognition. At the same time, the follow-up device 50 can also obtain the corresponding interview question information from the system server to associate the candidate's answer content with the corresponding interview question.

[0066] As Figure 6 shown, the above-mentioned interview question follow-up method provided by the present invention may further include step 602: inputting the answer content into a pre-trained follow-up model to use the follow-up model to judge the dimension that needs to be followed up.

[0067] As described above, the follow-up question model is an artificial intelligence (AI) model that can determine the follow-up dimensions to be questioned according to pre-trained follow-up rules. Through pre-training using the above training method, the follow-up rules of the follow-up question model can include: in response to the interview question being a first type of source question and the answer content lacking any first follow-up dimension, conducting an integrity follow-up on the missing first follow-up dimension; and in response to the interview question being a second type of source question and the answer content covering any second follow-up dimension, conducting a depth follow-up on the second follow-up dimension with insufficient answer depth.

[0068] As Figure 3 shown, in some embodiments, the first type of source question mainly assesses the comprehensive qualities of candidates based on the integrity of their answer content. The first type of source question can include multiple first follow-up dimensions. Each first follow-up dimension indicates a general competency of the candidate. The general competency includes, but is not limited to, soft skills such as communication skills, stress resistance, and teamwork skills, which are applicable to almost all positions. The interviewer can set multiple first follow-up dimensions for each first type of source question to comprehensively understand multiple general competencies of the candidate.

[0069] In contrast, the second type of source question mainly assesses the professional qualities of candidates based on the depth of their answer content. The second type of source question can include multiple second follow-up dimensions. Each second follow-up dimension indicates a position competency of the candidate. The position competency includes, but is not limited to, financial knowledge for financial positions, human resources knowledge for human resources positions, and programming techniques for programming positions. Each position competency can indicate a knowledge point to be assessed by the corresponding second type of source question. The interviewer can determine whether the candidate comprehensively understands each knowledge point of the corresponding professional skill based on whether the candidate's answer covers each second follow-up dimension. Further, the interviewer can set multiple assessment dimensions for each second follow-up dimension and further determine the depth of the candidate's mastery of the knowledge point based on whether the candidate's answer covers each assessment dimension. Furthermore, each second follow-up dimension can include multiple levels of assessment dimensions. Specifically, each second follow-up dimension can include multiple first-level assessment dimensions. Each first-level assessment dimension can include multiple second-level assessment dimensions. Each second-level assessment dimension can include multiple third-level assessment dimensions. And so on, conducting a depth follow-up on professional qualities from point to area. The interviewer can accurately determine the specific depth of the candidate's mastery of the knowledge point based on which level of assessment dimension the candidate's answer finally reaches.

[0070] In some embodiments, the trainer of the follow-up question model can configure both the interview question bank and the follow-up question bank in the memory 51 for the follow-up question model to query the corresponding follow-up dimensions according to the source questions involved in the interview and retrieve the corresponding follow-up questions from the follow-up question bank according to the follow-up dimensions to be questioned.

[0071] Specifically, when determining whether to conduct an integrity follow-up on the response content, the processor 52 of the follow-up model may first determine the first follow-up dimension involved based on the candidate's response content, and directly determine the first follow-up dimensions not involved in the response content as the missing first follow-up dimensions. In some preferred embodiments, the processor 52 may further determine the richness of the first follow-up dimensions involved in the response content. The richness can be evaluated in terms of whether the response content merely repeats the question, whether the response content is too broad, whether the response is too concise, and so on. In response to the richness of the response content for a follow-up dimension not meeting the standard, the processor 52 may also determine the non-compliant first follow-up dimension as the missing first follow-up dimension. After that, the processor 52 may retrieve the corresponding follow-up questions from the follow-up question library for the missing first follow-up dimensions to conduct an integrity follow-up.

[0072] For example, as Figure 3 shown, the asynchronous video interview system may pose a first type of source question 11 to the candidate, namely, "Please share the most difficult task you encountered in the past year. Specifically, what was the task content at that time? What difficulties did you encounter? How did you overcome them? What was the final task result?" The first type of source question 11 includes multiple first follow-up dimensions 111-114 such as background (Situation, S), content (Task, T), action (Action, A), and result (Result, R). Among them, each of the first follow-up dimensions 111-114 may correspond to one or more follow-up questions. If the candidate does not answer what the final task result was, the processor 52 may retrieve the follow-up question 1141 of "What was the result of the final task?" from the follow-up question library for the first follow-up dimension 114 of the task result (R) to conduct an integrity follow-up to guide the candidate to improve their response content.

[0073] When determining whether to conduct in-depth follow-up questions on the response content, the processor 52 implementing the follow-up question model may first determine the second follow-up dimension involved based on the candidate's response content, and then judge the assessment dimension further involved in the response content according to the second follow-up dimension involved in the response content. In response to the response content missing a second follow-up dimension, the processor 52 may determine that the candidate lacks knowledge of this point, so there is no need to conduct in-depth follow-up questions. In response to the response content involving a second follow-up dimension but not covering all assessment dimensions of this second follow-up dimension, the processor 52 may determine that the depth of the response content for this second follow-up dimension is insufficient. Therefore, for this second follow-up dimension with insufficient depth and the un-involved assessment dimensions, corresponding follow-up questions are retrieved from the follow-up question library to conduct in-depth follow-up questions. In some preferred embodiments, for embodiments where the second follow-up dimension includes multiple levels of assessment dimensions, after asking in-depth follow-up questions, the processor 52 may further obtain the candidate's supplementary response content from the asynchronous video interview system, and determine the secondary assessment dimension involved based on the obtained supplementary response content. In response to the supplementary response content involving a secondary assessment dimension but not covering all tertiary assessment dimensions of this secondary assessment dimension, the processor 52 may determine that the depth of the response content for this secondary assessment dimension is insufficient. Therefore, for this secondary assessment dimension with insufficient depth and the un-involved tertiary assessment dimensions, corresponding follow-up questions are retrieved from the follow-up question library to conduct further in-depth follow-up questions.

[0074] For example, Figure 3As shown, the asynchronous video interview system can pose a second - type source question 21 to candidates for the Java engineer position, namely, "Please introduce in detail the core components of the Java microservices architecture Spring Cloud." This second - type source question 21 includes multiple second - follow - up dimensions 211 - 215 such as the Eureka component, Ribbon component, Feign component, Zuul component, etc. Among them, each second - follow - up dimension 211 - 215 can include multiple assessment dimensions 2111 - 2112, 2121 - 2122 such as component functions and usage methods. Each assessment dimension 2111 - 2112, 2121 - 2122 can further correspond to one or more follow - up questions 21111, 21121, 21211 - 21213. If a candidate only mentions the name of a component during the answering process, such as "The components included in Spring Cloud are Eureka", but does not elaborate on the function of the Eureka component in detail, then the processor 52 can retrieve the follow - up question 21111, "What are the main functions of the Eureka component", from the follow - up question library for the assessment dimension 2111 of the component function to conduct in - depth follow - up to guide the candidate to deepen their answer content. Further, if the supplementary answer content of the candidate only involves some function names (i.e., secondary assessment dimensions) of the Eureka component and does not elaborate on the principles of these functions (i.e., tertiary assessment dimensions) in detail, then the processor 52 can retrieve the follow - up question, "Please explain in detail the implementation principle of this function", from the follow - up question library for the function principle of the Eureka component to conduct further in - depth follow - up to guide the candidate to further deepen their answer content.

[0075] As described above, the follow - up model needs to judge the dimension to be followed up according to the Q&A semantics of the interview questions and the answering content. In some embodiments, the follow - up model can include a classification module for judging whether to follow up on the candidate's answering content. Specifically, the classification module can select a text classification model based on a deep neural network (CNN, RNN (LSTM, GRU), Bi - LSTM, Bi - GRU, etc.) or a text classification model based on a pre - trained model (GPT, BERT, ELMO and their variants).

[0076] When determining the dimensions that need to be probed, the processor 52 can first perform speech recognition on the interview video data uploaded by the candidate to obtain the response text A of the response content. After that, the processor 52 can form a question-and-answer text pair <Q, A> by combining the response text A provided by the candidate with the question text Q of the interview question, and input the question-and-answer text pair <Q, A> into a pre-trained classification module to use the classification module to determine whether probing is needed. The training steps and judgment rules of the classification module have been introduced in detail in the above embodiments of training the probing model and will not be elaborated here. If the binary classification label output by the classification module is 1, it means that the candidate's response content is not complete enough or lacks depth, and probing is needed. On the contrary, if the binary classification label output by the classification module is 0, it means that the candidate's response content is already complete enough and does not involve the problem of lack of depth, and no further probing is needed.

[0077] In some embodiments, the probing model may further include a selection module for selecting the dimensions that need to be probed. Specifically, if the classification module determines that the candidate's response content needs to be probed, the processor 52 can further input the above response text A and question text Q into an encoder respectively to obtain the corresponding response semantic representation vector repA and question semantic representation vector repQ. The encoder can be built based on one of a convolutional neural network (CNN), a recurrent neural network (RNN, LSTM, GRU, Bi-LSTM, Bi-GRU, etc.) or a pre-trained model (ELMO, GPT, BERT, etc.). After that, the processor 52 can input the response semantic representation vector repA and question semantic representation vector repQ into the selection module to use the selection module to select the dimensions that need to be probed.

[0078] In some preferred embodiments, in order to further improve the semantic understanding ability of the selection module for the question-and-answer content, the processor 52 can perform further question-and-answer interaction on the obtained response semantic representation vector repA and question semantic representation vector repQ to obtain the corresponding question-and-answer relevance representation vector repQA. After that, the processor 52 can perform weighted fusion on the response semantic representation vector repA, the corresponding question semantic representation vector repQ and the corresponding question-and-answer relevance representation vector repQA of each response sample respectively to obtain the final relevance representation vector repFinal corresponding to each response sample, so as to map repQ, repA, and repQA to the same high-dimensional space for better classification.

[0079] After that, the processor 52 can input the final relevance representation vector repFinal of the candidate's answer content into the selection module to use the selection module to select the dimensions that need to be probed. The training steps and selection rules of the selection module have been introduced in detail in the above embodiments of training the probing model and will not be elaborated here. The trained selection module can output the corresponding label set (C0, C1, C2, C3) according to the input final relevance representation vector repFinal to more accurately indicate the dimensions that need to be probed.

[0080] For example, for Figure 3 the four probing dimensions 111-114 of STAR of the first type of source question 11 shown, the processor 52 can use a label set (C0, C1, C2, C3) with a dimension of 4 to indicate the dimensions that need to be probed. Specifically, C0 indicates whether the background (S) dimension 111 needs to be probed. If the S dimension 111 does not need to be probed, the selection module can output C0 = 0. On the contrary, if the S dimension 111 needs to be probed, the selection module can output C0 = 1. Similarly, C1, C2, and C3 can respectively indicate whether the first probing dimensions 112-114 corresponding to TAR need to be probed. The trained selection module can output the corresponding label set (C0, C1, C2, C3) according to the input answer semantic representation vector repA and its corresponding question semantic representation vector repQ to indicate the dimensions that need to be probed.

[0081] As Figure 6 shown, the above method for probing interview questions provided by the present invention may further include step 603: using the probing model to retrieve the corresponding probing questions from the probing question bank according to the probing dimensions that need to be probed for probing.

[0082] As described above, in some embodiments of the present invention, the interview question bank may include multiple first-type source questions and multiple second-type source questions for respectively assessing the comprehensive quality and professional quality of candidates. In some embodiments, a first-type source question may include K first probing dimensions for evaluating the completeness of the candidate's answer content. The processor 22 can configure S probing questions for each probing dimension for the probing model to randomly select one probing question from them to conduct a completeness probe on the candidate. At this time, this first-type source question may correspond to K*S probing questions. In some embodiments, the processor 22 can configure more than 20 probing questions (i.e., S>20) for each first probing dimension to prevent candidates from deliberately preparing for the same probing questions using the experience of multiple interviews to cover up their true situation.

[0083] Similarly, a second type of source question may include K1 second follow-up dimensions for evaluating the completeness of the candidate's answer content. Each second follow-up dimension may further include K2 assessment dimensions for evaluating the depth of the candidate's answer content in that second follow-up dimension. The processor 22 may configure S follow-up questions for each assessment dimension, for the follow-up model to randomly select one follow-up question from them to conduct in-depth follow-up on the candidate. At this time, defining K = K1 * K2, this second type of source question may also correspond to K * S follow-up questions. Similarly, the processor 22 may also configure more than twenty follow-up questions (i.e., S > 20) for each second follow-up dimension to prevent candidates from deliberately preparing for the same follow-up questions using the experience of multiple interviews to cover up their true situation.

[0084] As Figure 4 shown, in some embodiments, the follow-up question library may be constructed in the dictionary form of <source question, K * S follow-up questions>. This follow-up question library dictionary constructs source question entries in units of source questions. By including multiple follow-up dimensions of each source question and multiple follow-up questions of each follow-up dimension in the corresponding source question entry, the K * S follow-up questions can be associated with the corresponding source questions.

[0085] For example, the entry of the first type of source question 11 may include four first follow-up dimensions 111 to 114. Each follow-up dimension (e.g., follow-up dimension 111) may correspond to three follow-up questions 1111 to 1113. The processor 22 may construct the entry of the first type of source question 11 in the form of <the first type of source question 11, 4 * 3 follow-up questions>, and construct the follow-up question library dictionary by including the entries of all source questions. During an asynchronous video interview, the processor 52 implementing the follow-up model may query the corresponding follow-up dimensions 111 to 114 according to the first type of source question 11 raised in the interview, and randomly select one follow-up question 1111 from multiple relevant follow-up questions in the follow-up question library according to the first follow-up dimension 111 that needs to be followed up, to conduct completeness follow-up on the candidate.

[0086] Based on the above description, it can be seen that during an asynchronous video interview, by judging the completeness and depth of the candidate's answer content according to multiple follow-up dimensions of the interview questions, the follow-up model adopted by the present invention can provide corresponding follow-up questions for the deficiencies in the answer content to help the candidate supplement the answer content, so as to ensure that the interviewer can obtain sufficient information to judge the matching degree between the candidate and the recruitment position.

[0087] Although the methods described above are illustrated and described as a series of acts for simplicity of explanation, it should be understood and appreciated that the methods are not limited by the order of the acts, since according to one or more embodiments, some acts may occur in different orders and / or concurrently with other acts not illustrated and described herein but understood by those skilled in the art.

[0088] Those skilled in the art will appreciate that information, signals, and data can be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or optical particles, or any combination thereof.

[0089] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithmic steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

[0090] The various illustrative logical modules and circuits described in connection with the embodiments disclosed herein can be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0091] Although the processors 22, 52 described in the above embodiments can be implemented by a combination of software and hardware, it is understood that these processors 22, 52 can also be implemented individually in software or hardware. For hardware implementation, the processors 22, 52 can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for performing the above functions, or a selected combination of the above devices. For software implementation, the processors 22, 52 can be implemented by independent software modules such as procedures and functions running on a general-purpose chip, where each module can perform one or more of the functions and operations described herein.

[0092] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A training method for a questioning model, characterized in that, Including: Constructing a follow-up question bank according to at least one follow-up dimension of each source question in the interview question bank, where the interview question bank includes multiple source questions, and each follow-up dimension corresponds to at least one follow-up question in the follow-up question bank; According to each follow-up dimension of each source question, annotating the follow-up results of the answer integrity and answer depth of multiple answer samples, where the follow-up result annotation indicates whether each follow-up dimension needs to be followed up; Constructing a follow-up model, which is suitable for judging the follow-up dimension that needs to be followed up according to the trained follow-up rules, and retrieving the corresponding follow-up question from the follow-up question bank according to the follow-up dimension that needs to be followed up; And Training the follow-up model to judge the follow-up dimension that needs to be followed up according to the answer content of the interview candidate according to the multiple answer samples and their corresponding follow-up result annotations.

2. The training method according to claim 1, wherein The interview question bank includes two types of source questions, where the first type of source questions includes multiple first follow-up dimensions for indicating the answer integrity, and the second type of source questions includes multiple second follow-up dimensions for indicating the answer depth. The follow-up rules include: If the interview question is the first type of source question and the answer content of the interview candidate is missing any of the first follow-up dimensions, conduct an integrity follow-up on the missing first follow-up dimension; and If the interview question is the second type of source question and the answer content of the interview candidate involves any of the second follow-up dimensions, conduct a depth follow-up on the second follow-up dimension with insufficient answer depth involved.

3. The training method according to claim 2, characterized in that, The first type of source questions includes general competency questions for assessing the general competency of the interview candidate. The steps for conducting the integrity follow-up include: According to the answer content of the interview candidate, determine the first follow-up dimension involved in the answer content, and determine the missing first follow-up dimension as the one not involved; For the involved first follow-up dimension, judge the richness of the answer content; and In response to the richness not meeting the standard, also determine the non-compliant first follow-up dimension as the missing first follow-up dimension.

4. The training method according to claim 2, wherein The second type of source questions includes position competency questions for assessing the professional skills of the interview candidate. Each second follow-up dimension includes multiple assessment dimensions for indicating multiple professional knowledge points to evaluate the answer depth of the interview candidate. The steps for conducting the depth follow-up include: According to the answer content of the interview candidate, determine the second follow-up dimension involved in the answer content; For the involved second follow-up dimension, judge the further involved assessment dimension of the answer content; and In response to the answer content having an un-involved assessment dimension, conduct the depth follow-up for the un-involved assessment dimension.

5. The training method according to claim 4, wherein The second follow-up dimension includes multi-level assessment dimensions, where each second follow-up dimension includes multiple first-level assessment dimensions, each first-level assessment dimension includes multiple second-level assessment dimensions, and each second-level assessment dimension includes multiple third-level assessment dimensions. The steps for conducting the depth follow-up further include: In response to the existence of a first-level assessment dimension not covered in the response content, conduct the in-depth probing for the unaddressed first-level assessment dimension; Determine the involved second-level assessment dimensions based on the supplementary response content of the interview candidate to the probing questions; For the involved second-level assessment dimensions, judge the further involved third-level assessment dimensions in the supplementary response content; and In response to the existence of a third-level assessment dimension not covered in the supplementary response content, conduct further in-depth probing for the unaddressed third-level assessment dimension.

6. The training method according to claim 2, wherein The steps of performing the annotation of the probing results include: Based on the richness of each first probing dimension of the first type of source questions in the response sample, annotate the missing first probing dimensions of the response sample; and Based on the involvement of each assessment dimension of each second probing dimension of the second type of source questions in the response sample, annotate the second probing dimensions with insufficient response depth involved in the response sample, and annotate the assessment dimensions not involved in the response sample.

7. The training method according to claim 1, characterized in that, The source questions include K of the probing dimensions, and each of the probing dimensions corresponds to S probing questions. The steps of constructing the probing question bank include: Construct the probing question bank in the form of a dictionary <source question, K*S probing questions> for the probing model to query the corresponding probing dimension according to the source question raised in the interview and retrieve the corresponding probing question from the probing question bank according to the probing dimension that needs to be probed.

8. The training method according to claim 1, characterized in that The probing model includes a classification module. The steps of training the probing model include: Respectively form multiple question-and-answer text pairs by combining the response text of the multiple response samples with the question text of the corresponding source questions; and Use the multiple question-and-answer text pairs as inputs and the indication of whether the corresponding probing result annotation indicates the probing dimension that needs to be probed as the output, and train the classification module to judge whether further probing is needed based on the response content of the interview candidate.

9. The training method according to claim 8, wherein, The probing model further includes a selection module. The steps of training the probing model further include: Determine the corresponding response semantic representation vector based on the response text and determine the corresponding question semantic representation vector based on the question text; and Use the response semantic representation vectors of the multiple response samples and their corresponding question semantic representation vectors as inputs and the probing dimension that needs to be probed indicated by the corresponding probing result annotation as the output, and train the selection module to judge the probing dimension that needs to be probed based on the response content of the interview candidate.

10. The training method according to claim 9, characterized in that, The steps of training the selection module include: Interact with each of the response semantic representation vectors and their corresponding question semantic representation vectors to obtain the corresponding question-and-answer correlation representation vectors; Perform weighted fusion on each of the response semantic representation vectors, the corresponding question semantic representation vectors, and the corresponding question-and-answer correlation representation vectors to obtain the corresponding final correlation representation vectors; and Use the final correlation representation vectors of the multiple response samples as inputs and the probing dimension that needs to be probed indicated by the corresponding probing result annotation as the output, and train the selection module to judge the probing dimension that needs to be probed based on the response content of the interview candidate.

11. A method for following up interview questions, characterized in that, Include: Collect the answer content of the interview candidate to the interview questions. Among them, the interview questions are source questions stored in the interview question bank. The interview question bank includes multiple such source questions. Each source question includes multiple follow-up dimensions, and each follow-up dimension corresponds to at least one follow-up question in the follow-up question bank; Input the answer content into a pre-trained follow-up model to use the follow-up model to judge the follow-up dimensions that need to be followed up. Among them, the follow-up model judges the answer integrity and answer depth of the answer content according to each follow-up dimension of the interview question, and judges whether each follow-up dimension needs to be followed up according to the answer integrity and the answer depth; and Use the follow-up model to retrieve the corresponding follow-up questions from the follow-up question bank according to the follow-up dimensions that need to be followed up for follow-up.

12. The interrogation method according to claim 11, wherein The interview question bank includes two types of source questions. Among them, the first type of source questions includes multiple first follow-up dimensions for indicating the answer integrity, and the second type of source questions includes multiple second follow-up dimensions for indicating the answer depth. The steps for judging whether each follow-up dimension needs to be followed up include: In response to the interview question being the first type of source question and the answer content lacking any of the first follow-up dimensions, conduct integrity follow-up on the missing first follow-up dimension; and In response to the interview question being the second type of source question and the answer content covering any of the second follow-up dimensions, conduct depth follow-up on the second follow-up dimensions with insufficient answer depth covered.

13. The questioning method according to claim 12, wherein The first type of source questions includes general competency questions for assessing the general competency of the interview candidate. The steps for conducting the integrity follow-up include: Determine the first follow-up dimensions covered by the answer content according to the answer content, and determine the uncovered first follow-up dimensions as the missing first follow-up dimensions; For the covered first follow-up dimensions, judge the richness of the answer content; and In response to the richness not meeting the standard, also determine the non-compliant first follow-up dimensions as the missing first follow-up dimensions.

14. The follow-up method according to claim 12, characterized in that, The second type of source questions includes position competency questions for assessing the professional skills of the interview candidate. Among them, each second follow-up dimension includes multiple assessment dimensions for indicating multiple professional knowledge points to evaluate the answer depth of the interview candidate. The steps for conducting the depth follow-up include: Determine the second follow-up dimensions covered by the answer content according to the answer content; For the covered second follow-up dimensions, judge the further assessment dimensions covered by the answer content; and In response to the answer content having uncovered assessment dimensions, conduct the depth follow-up for the uncovered assessment dimensions.

15. The follow-up method according to claim 14, characterized in that, The second follow-up dimension includes multi-level assessment dimensions. Among them, each second follow-up dimension includes multiple first-level assessment dimensions, each first-level assessment dimension includes multiple second-level assessment dimensions, and each second-level assessment dimension includes multiple third-level assessment dimensions. The steps for conducting the depth follow-up further include: In response to the answer content having uncovered first-level assessment dimensions, conduct the depth follow-up for the uncovered first-level assessment dimensions; Collect the supplementary answer content of the interviewed candidate to the follow-up questions, and determine the secondary assessment dimensions involved according to the supplementary answer content; For the secondary assessment dimensions involved, judge the tertiary assessment dimensions further involved in the supplementary answer content; and In response to the existence of tertiary assessment dimensions not involved in the supplementary answer content, conduct further in-depth follow-up questions for the un-involved tertiary assessment dimensions.

16. The follow-up method according to claim 11, characterized in that, The source question includes K of the follow-up dimensions, and each of the follow-up dimensions corresponds to S follow-up questions. The follow-up question library is constructed in the form of a dictionary <source question, K*S follow-up questions>. The steps of retrieving the corresponding follow-up questions from the follow-up question library for follow-up include: Query the corresponding source question according to the interview question to determine the K follow-up dimensions included in the corresponding entry; According to the follow-up dimension that needs to be followed up, determine S follow-up questions from the corresponding entry of the interview question; and Randomly select one follow-up question from the S follow-up questions for follow-up.

17. The follow-up method according to claim 11, characterized in that, The follow-up model includes a classification module. The steps of judging the follow-up dimension that needs to be followed up include: Identify the answer text of the answer content, and form a Q&A text pair with the question text of the interview question; and Input the Q&A text pair into the pre-trained classification module to use the classification module to judge whether follow-up is needed.

18. The questioning method according to claim 17, wherein The follow-up model further includes a selection module. The steps of judging the follow-up dimension that needs to be followed up further include: Determine the corresponding answer semantic representation vector according to the answer text, and determine the corresponding question semantic representation vector according to the question text; and Input the answer semantic representation vector and the question semantic representation vector into the pre-trained selection module to use the selection module to judge the follow-up dimension that needs to be followed up.

19. The inquiring method according to claim 18, characterized in that, The steps of using the selection module to judge the follow-up dimension that needs to be followed up include: Interact the answer semantic representation vector and the question semantic representation vector to obtain a Q&A relevance representation vector; Perform weighted fusion on the answer semantic representation vector, the question semantic representation vector and the Q&A relevance representation vector to obtain a final relevance representation vector; and Use the selection module to judge the follow-up dimension that needs to be followed up according to the final relevance representation vector.

20. A training device for an inquiry model, characterized in that Include: A memory; And A processor, the processor is connected to the memory and is configured to implement the training method of the follow-up model as described in any one of claims 1 to 10.

21. An interrogation device for interview questions, characterized in that, Include: A memory; And A processor, the processor is connected to the memory and is configured to implement the follow-up method of the interview question as described in any one of claims 11 to 19.

22. A computer-readable storage medium having computer instructions stored thereon, characterized in that, When the computer instructions are executed by the processor, the training method of the follow-up model as described in any one of claims 1 to 10 is implemented.

23. A computer-readable storage medium having computer instructions stored thereon, characterized in that, When the computer instructions are executed by the processor, the follow-up method of the interview question as described in any one of claims 11 to 19 is implemented.

Images