An interview comprehensive evaluation method based on reinforcement learning

Abstract

The application relates to the technical field of reinforcement learning, in particular to an interview comprehensive evaluation method based on reinforcement learning, which comprises the following steps: determining an interviewee category according to information matching degree and working time reference value, and determining to perform sequential matching or direct matching according to the interviewee category; in the sequential matching, determining target test questions corresponding to each problem dimension based on test question matching degree, determining an analysis state of to-be-generated reference test questions according to a quantity reference value and a span coefficient, and determining to perform adjacent analysis generation or directly generating reference test questions according to a knowledge span value according to the analysis state; in the direct matching, determining a generation state according to a request difficulty value and a maximum matching coefficient, and determining to perform template matching or template combination to generate test questions according to the generation state; and determining whether the interview is qualified according to an interview evaluation coefficient. The application can improve interview evaluation accuracy.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and in particular to a comprehensive interview evaluation method based on reinforcement learning. Background Technology

[0002] In corporate recruitment, the interview is a core step in candidate screening. The accuracy, efficiency, and fairness of the evaluation directly determine the quality of talent selection, thus impacting the company's talent pool and development efficiency. Structured interviews suffer from inefficiency and fixed questions, failing to adapt to the dynamic matching needs between candidates and positions. Therefore, improving the accuracy of candidate-job matching and the precision of interview evaluations is a pressing issue for those skilled in the art.

[0003] Chinese Patent Publication No. CN115345591A discloses an intelligent interview method, intelligent interview device, and intelligent interview system, comprising: during an interview, after obtaining the interviewee's answer to the current interview question and determining the score for the current interview question, determining the next interview question from the interview paper, historical interview records, or an interview question bank based on the question score and the depth and breadth of the current interview question, and providing the determined next interview question to the interviewee for answering. In response to the determination that there is no next interview question, determining the interviewee's interview score based on the scores of each basic question in the interview paper. However, the above solution has the following problems: it does not design personalized assessment strategies for the differences in candidates' abilities, making it difficult to achieve accurate matching of people and positions and comprehensive, dynamic interview evaluation, resulting in poor accuracy of interview evaluation. Summary of the Invention

[0004] To address this issue, the present invention provides a comprehensive interview evaluation method based on reinforcement learning, which overcomes the problem in existing technologies that fail to design personalized assessment strategies for the differences in candidates' abilities, making it difficult to achieve accurate matching between people and positions and comprehensive and dynamic interview evaluation, resulting in poor accuracy in interview evaluation.

[0005] To achieve the above objectives, this invention provides a comprehensive interview evaluation method based on reinforcement learning, comprising:

[0006] Obtain information about the target job position and the resumes of several interviewees for the corresponding job position;

[0007] The categories of interviewees are determined based on the degree of information matching and reference values ​​for working hours, and then sequential matching or direct matching is performed based on the categories of interviewees.

[0008] In sequential matching, the target questions for each question dimension are determined based on the question matching degree. The analysis status of the reference questions to be generated is determined based on the quantity reference value and the span coefficient. Based on the analysis status, it is determined whether to generate reference questions through proximity analysis or directly generate reference questions based on the knowledge span value.

[0009] In the proximity analysis generation, generation is determined based on the accuracy of neighboring test items, either by a priority threshold or by the fluctuation dimension and information load index.

[0010] In direct matching, the generation status is determined based on the request difficulty value and the maximum matching coefficient, and template matching or template combination is performed based on the generation status to generate test questions;

[0011] The interview pass / fail status is determined based on the interview evaluation coefficient.

[0012] Furthermore, sequential matching is performed on interviewees whose information matching degree is greater than or equal to the preset information matching degree or whose working time reference value is greater than or equal to the preset working time reference value.

[0013] Furthermore, for two types of interviewees, whose information matching degree is less than the preset information matching degree and whose working time reference value is less than the preset working time reference value, direct matching is performed;

[0014] The categories of interviewees include Category I interviewees and Category II interviewees.

[0015] Furthermore, for the reference test questions to be generated whose analysis status is that the quantity reference value is greater than or equal to the preset quantity reference value and the span coefficient is greater than or equal to the preset span coefficient, a proximity analysis is performed to generate them;

[0016] Proximity analysis is in progress.

[0017] For reference questions to be generated where the accuracy of neighboring questions is greater than or equal to the preset accuracy of neighboring questions, they are generated according to a priority threshold.

[0018] For reference questions to be generated where the accuracy of neighboring questions is lower than the preset accuracy of neighboring questions, they are generated based on the fluctuation dimension and information load index.

[0019] Furthermore, it is generated based on the fluctuation dimension and information load index, including:

[0020] For reference test questions to be generated that have a fluctuation dimension greater than or equal to a preset fluctuation dimension or an information load index greater than or equal to a preset information load index, they are generated based on the evaluation threshold.

[0021] For reference test questions to be generated that have a fluctuation dimension smaller than the preset fluctuation dimension and an information load index smaller than the preset information load index, the generation of the test questions is stopped based on the proximity error rate or dynamically generated based on the learning curve reference value.

[0022] Furthermore, for reference test questions to be generated that have a neighbor error rate lower than the preset neighbor error rate, they are dynamically generated based on the learning curve reference value;

[0023] When dynamically generated based on the learning curve reference value

[0024] For reference test questions to be generated whose learning curve reference value is greater than or equal to the preset learning curve reference value, reference test questions are generated based on the intensity range of the first preset test point.

[0025] For reference questions to be generated where the learning curve reference value is less than the preset learning curve reference value, reference questions are generated based on the intensity range of the second preset examination point.

[0026] Furthermore, for reference questions to be generated where the analysis status shows that the quantity reference value is less than the preset quantity reference value or the span coefficient is less than the preset span coefficient, reference questions are directly generated based on the knowledge span value.

[0027] Furthermore, when the generated status is that the request difficulty value is greater than or equal to the preset request difficulty value and the maximum matching coefficient is greater than or equal to the preset maximum matching coefficient, template matching is performed;

[0028] The difficulty value of the request is determined based on the number of interviewees of a certain type and the estimated generation time.

[0029] Furthermore, when the generated status is that the request difficulty value is less than the preset request difficulty value or the maximum matching coefficient is less than the preset maximum matching coefficient, template combination is performed.

[0030] Furthermore, for interviewees whose interview evaluation coefficient is less than the preset interview evaluation coefficient, the interview is deemed unqualified;

[0031] For interviewees whose interview evaluation coefficient is greater than or equal to the preset interview evaluation coefficient, the interview is deemed qualified.

[0032] Compared with the prior art, the beneficial effects of the present invention are that, in the technical solution of the present invention, the basic matching level between the interviewee and the target position in terms of skill suitability and experience accumulation is effectively reflected by the information matching degree and working time reference value. Then, the interviewee category is determined according to the information matching degree and working time reference value, which helps to avoid the bias of classifying the object based solely on subjective judgment. Furthermore, the sequential matching or direct matching is determined according to the interviewee category, making the interview question generation strategy more in line with the actual application scenario, which helps to balance the accuracy and efficiency of the interview, and ultimately improves the accuracy of the interview evaluation.

[0033] Furthermore, this invention effectively reflects the degree to which the number of generated interview questions meets the standard and the diversity of content coverage under the target question dimension through quantitative reference values ​​and span coefficients. Then, based on the analysis status, it determines whether to generate reference questions through proximity analysis or directly generate reference questions based on knowledge span values. This helps to avoid the blindness of question generation and ensures the comprehensiveness and efficiency of interview assessment.

[0034] Furthermore, this invention effectively reflects the degree to which the interviewee's answer quality on adjacent questions matches the standard answer by using the accuracy of adjacent questions. Then, based on the accuracy of adjacent questions, questions are adaptively generated according to priority thresholds or according to fluctuation dimensions and information load indices. This facilitates dynamic calibration of question generation, avoiding blindly selecting questions without considering the foundation of previous assessments, and accurately adjusting the direction of subsequent assessments based on the performance of previous answers. This improves the fit between questions and interview objectives, ultimately enhancing the scientific rigor and accuracy of interview evaluation.

[0035] Furthermore, this invention effectively reflects the task pressure of generating current interview questions and the accuracy of matching with the templates to be selected by requesting difficulty value and maximum matching coefficient. Then, it determines whether to perform template matching or template combination based on the generation status, which is conducive to achieving a dynamic balance between interview question generation efficiency and adaptability. This reduces the time and manpower costs of question generation and improves the effectiveness of interview assessment through accurately matched questions. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the comprehensive interview evaluation method based on reinforcement learning according to the present invention;

[0037] Figure 2 This is a flowchart illustrating the process of determining whether to perform direct matching or sequential matching based on information matching degree and working time reference value in this invention.

[0038] Figure 3 This is a flowchart illustrating the process of determining template matching or template combination based on the generation state in this invention;

[0039] Figure 4 This is a flowchart for determining whether an interview is satisfactory based on an interview evaluation coefficient, as per the present invention. Detailed Implementation

[0040] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0041] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0042] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0043] Please see Figures 1 to 4 As shown, this invention provides a comprehensive interview evaluation method based on reinforcement learning, including:

[0044] Obtain information about the target job position and the resumes of several interviewees for the corresponding job position;

[0045] The categories of interviewees are determined based on the degree of information matching and reference values ​​for working hours, and then sequential matching or direct matching is performed based on the categories of interviewees.

[0046] In sequential matching, the target questions for each question dimension are determined based on the question matching degree. The analysis status of the reference questions to be generated is determined based on the quantity reference value and the span coefficient. Based on the analysis status, it is determined whether to generate reference questions through proximity analysis or directly generate reference questions based on the knowledge span value.

[0047] In the proximity analysis generation, generation is determined based on the accuracy of neighboring test items, either by a priority threshold or by the fluctuation dimension and information load index.

[0048] In direct matching, the generation status is determined based on the request difficulty value and the maximum matching coefficient, and template matching or template combination is performed based on the generation status to generate test questions;

[0049] The interview pass / fail status is determined based on the interview evaluation coefficient.

[0050] The application scenario of this invention is to evaluate the interview performance of interviewees through reinforcement learning. By using reinforcement learning technology to determine the category of interviewees, generate test questions, and determine whether the interview is qualified, it realizes intelligent classification of candidate abilities, adaptive generation of test questions, and recruitment determination, which significantly improves interview efficiency and evaluation accuracy.

[0051] This invention includes several historical records. Each historical record contains at least one evaluation of an interviewee's performance using reinforcement learning, including quantitative reference values, span coefficients, fluctuation dimensions, and information load indexes. Each historical record also has a corresponding pass / fail marker. The pass / fail marker indicates whether the evaluation process using reinforcement learning meets the user's needs. The pass / fail marker can be manually recorded. It is understood that the user can determine whether the evaluation process using reinforcement learning meets the user's needs based on self-defined indicators. These self-defined indicators can be, but are not limited to, the number of anomalies, which will not be elaborated here. The number of anomalies refers to the number of times a disqualified interviewee was mistakenly judged as qualified.

[0052] The target job information includes, but is not limited to, the job description, job requirements, and work location for each individual position. The target job information includes several interviewees who will be interviewed for the target position. The resume information for each interviewee includes, but is not limited to, personal information, job objective, educational background, and work experience. It should be noted that each work experience must clearly state the job title, job function, and the "start time - end time" associated with the job title (accurate to the month, in the format "YYYY(year).MM(month)-YYYY(year).MM(month)").

[0053] This invention includes an interview question library containing a number of interview questions. Each interview question corresponds to a question dimension and a standard answer. Question dimensions include, but are not limited to, self-awareness, professional skills, and job awareness. Interview questions related to self-awareness include, but are not limited to, "Why did you choose this position?" and "If you were to summarize yourself using 'strengths-areas for improvement-goals', what would you say?" Interview questions related to professional skills include, but are not limited to, "Please describe the complete process of data cleaning you implemented using Python?" and "Explain the core principles of Spring Boot, which you are familiar with?" Interview questions related to job awareness include, but are not limited to, "If you were to give us an improvement suggestion for our product, what would you suggest?" and "What do you consider to be the most challenging part of the daily work of this position?" These are contents that are easily understood by those skilled in the art and will not be elaborated upon further.

[0054] The present invention also provides a template library to be selected, which contains several templates to be selected. Each template to be selected corresponds to several interview questions, and each interview question is marked with a corresponding question dimension. This is content that is easy for those skilled in the art to understand, and will not be elaborated on in detail.

[0055] Specifically, sequential matching is performed on interviewees whose information matching degree is greater than or equal to the preset information matching degree or whose working time reference value is greater than or equal to the preset working time reference value.

[0056] Specifically, direct matching will be carried out for two types of interviewees: those whose information matching degree is less than the preset information matching degree and whose working time reference value is less than the preset working time reference value.

[0057] The categories of interviewees include Category I interviewees and Category II interviewees.

[0058] Specifically, each keyword in the target job information is detected and recorded as job keywords. For each interviewee corresponding to the target job information, the interviewee is recorded as the target interviewee. Each keyword in the resume information of the target interviewee is detected and recorded as resume keywords. The keywords in the target job information and resume information are extracted using NLP algorithms, which are common techniques used by those skilled in the art and will not be elaborated on in detail.

[0059] The information matching degree of the target interviewee is the average of the maximum keyword similarity corresponding to the keywords of each position;

[0060] The maximum keyword similarity corresponding to a single job keyword is the maximum value among the keyword similarities between the job keyword and each resume keyword. For a single job keyword and a single resume keyword, the word2vec model is used to convert the two keywords into word vectors. The cosine similarity between the two word vectors is recorded as the keyword similarity between the job keyword and the resume keyword. The value range of the keyword similarity is [0,1].

[0061] Each job title contained in the target job information is recorded as a reference job title. The job title in the resume information of the target interviewee whose keyword similarity with any reference job title is greater than the preset keyword similarity is recorded as a matching job title. The sum of the working hours corresponding to each matching job title in the resume information of the target interviewee is recorded as the reference value of the working time of the target interviewee, in months.

[0062] Users can determine the preset values ​​for keyword similarity, information matching, and working time reference based on their actual application scenarios. The greater the user's need for improved accuracy in matching test questions, the higher the preset keyword similarity value and the lower the preset information matching and working time reference values. One preset value for these three values ​​is provided: preset keyword similarity is 0.7, preset information matching is 0.6, and preset working time reference is 12 months.

[0063] Understandably, information matching degree and working time reference value effectively reflect the basic suitability potential between interviewees and target positions. For interviewees whose information matching degree is greater than or equal to the preset information matching degree or whose working time reference value is greater than or equal to the preset working time reference value, it indicates that their core skills or relevant practical experience required for the position have reached the basic suitability standard and are worthy of further in-depth assessment, so sequential matching is performed. For interviewees whose information matching degree is less than the preset information matching degree and whose working time reference value is less than the preset working time reference value, it indicates that their skills are not well matched with the core requirements of the position and their relevant practical experience is limited, so their basic suitability needs to be verified, and direct matching is performed.

[0064] Sequential matching includes: generating analysis for each question dimension; when generating analysis for a single question dimension, the question dimension is recorded as the target question dimension; the target question corresponding to the target question dimension is determined based on the question matching degree; and the status analysis is performed on each reference question to be generated in the order of generation from first to last. When performing status analysis on a single reference question to be generated, the analysis status of the reference question to be generated is determined based on the quantity reference value and the span coefficient. Based on the analysis status, it is determined whether to generate a reference question through proximity analysis or based on the knowledge span value. Then, the status analysis is continued for other reference questions to be generated that have not yet generated reference questions, until all reference questions to be generated have generated corresponding reference questions.

[0065] There are no restrictions on the specific order in which the various question dimensions are generated; users can determine this themselves.

[0066] For a single interviewee, the target question for the target question dimension is the first interview question generated for that interviewee in the target question dimension, and the reference questions are the other interview questions corresponding to the target question dimension after the first interview question generated for that interviewee in the target question dimension.

[0067] For a single interviewee, the matching degree of a single interview question = the number of keywords in the interview question whose keyword similarity to any keyword in the interviewee's resume is greater than the preset keyword similarity × first weight coefficient + the number of keywords in the interview question whose keyword similarity to any keyword in the job posting is greater than the preset keyword similarity × second weight coefficient; both the first weight coefficient and the second weight coefficient are 0.5;

[0068] When determining the target questions for the target question dimension based on the question matching degree, the interview question with the highest question matching degree among all the interview questions corresponding to the target question dimension in the interview question bank is taken as the target question.

[0069] The number of reference questions to be generated for a single question dimension, n, is the smallest integer greater than or equal to n0, and n0 is the average number of reference questions for each type of interviewee in the historical records that can meet the user's needs.

[0070] The analysis states include a first analysis state and a second analysis state. The first analysis state is when the quantity reference value is greater than or equal to the preset quantity reference value and the span coefficient is greater than or equal to the preset span coefficient. The second analysis state is when the quantity reference value is less than the preset quantity reference value or the span coefficient is less than the preset span coefficient.

[0071] Understandably, questions are generated based on the matching degree and analysis status. First, target questions are generated, and then reference questions are generated one by one. The reference questions to be generated are those that have not yet been generated.

[0072] For a single reference question to be generated corresponding to the target question dimension, the reference question to be generated is denoted as the target reference question to be generated. The interview questions corresponding to the target question dimension that have been generated before the target reference question to be generated are denoted as the target generated questions. The reference value for the number of target reference questions to be generated is the number of target generated questions.

[0073] The formula for calculating the span coefficient K corresponding to the target reference test questions to be generated is:

[0074] ;

[0075] Where K is the span coefficient, m is the number of questions generated for the target, i,j=1,2,...,m, and i≠j. Let the similarity reference value be the corresponding generated question for the i-th target and the generated question for the j-th target. Let the "interview question with more keywords" in the two interview questions be recorded as the benchmark question. The similarity reference value is the number of keywords in the benchmark question whose keyword similarity with any keyword in the other interview question is greater than the preset keyword similarity.

[0076] Specifically, for reference test questions to be generated whose analysis status is that the quantity reference value is greater than or equal to the preset quantity reference value and the span coefficient is greater than or equal to the preset span coefficient, a proximity analysis is performed to generate the test questions.

[0077] Proximity analysis is in progress.

[0078] For reference questions to be generated where the accuracy of neighboring questions is greater than or equal to the preset accuracy of neighboring questions, they are generated according to a priority threshold.

[0079] For reference questions to be generated where the accuracy of neighboring questions is lower than the preset accuracy of neighboring questions, they are generated based on the fluctuation dimension and information load index.

[0080] Specifically, the values ​​of the preset quantity reference value and the preset span coefficient can be determined by the user according to the actual application scenario. The greater the user's need to improve the flexibility of the test questions, the larger the values ​​of the preset quantity reference value and the preset span coefficient should be. A method for determining the preset quantity reference value and the preset span coefficient is provided, which detects the historical records generated by the user's proximity analysis, and records the average value of the quantity reference value and the average value of the span coefficient corresponding to the historical records that can meet the user's needs as the preset quantity reference value and the preset span coefficient, respectively.

[0081] The text includes, but is not limited to, the content of the answers to interview questions, the standard answers to interview questions, and the interview questions themselves. For any text, the word vectors corresponding to each keyword in the text are determined by the word2vec model, the weight of each keyword in the text is determined by the TF-IDF algorithm, the word vector is multiplied by the corresponding TF-IDF weight, and then the average of all weighted word vectors is taken to obtain a text vector with the same dimension as the word vectors. In addition, there are other technical means commonly used by those skilled in the art, which will not be elaborated on in detail.

[0082] For a single reference question to be generated, the cosine similarity between the text vector corresponding to the answer content of an interview question that was generated earlier than the reference question to be generated and the text vector corresponding to the standard answer is recorded as the neighboring question accuracy.

[0083] The user can determine the preset value of the accuracy of the neighboring questions based on the actual application scenario. The smaller the preset value of the accuracy of the neighboring questions, the greater the user's need to generate questions based on the priority threshold. One preset value of the accuracy of the neighboring questions is provided, which is 0.6.

[0084] For a single interviewee, the interview questions generated for the target question dimension are recorded as the first question, and the other questions in the interview question library corresponding to the target question dimension, excluding the first question, are recorded as the second question;

[0085] When generating based on priority thresholds, for a single reference question to be generated, the second question with the highest priority threshold is used as the reference question for that reference question to be generated.

[0086] The priority threshold for a single interview question = the number of interviewees who generated the interview question in the history that can meet the user's historical needs / the total number of interviewees who generated the interview question in the entire history.

[0087] Understandably, the quantity reference value and span coefficient can accurately reflect the sufficiency of the number of generated interview questions and the diversity of content coverage under the target question dimension. When in the first analysis state, it means that the number of generated questions has met the basic assessment requirements, and the content coverage is sufficiently dispersed with no obvious repetition. At this time, the proximity analysis generation strategy can dynamically adjust subsequent questions based on the performance of previous interview questions, thereby achieving a refined and in-depth exploration of the interviewee's abilities. When in the second analysis state, it indicates that the number of generated questions has not met the basic assessment standards or the content coverage is too concentrated, with blind spots in the assessment. It is necessary to use the knowledge span value to filter out questions with large differences from the generated questions to improve the comprehensiveness of the assessment dimension.

[0088] The accuracy of adjacent questions effectively reflects the degree to which the interviewee's answers to adjacent questions match the standard answers. When the accuracy of adjacent questions is greater than or equal to the preset accuracy of adjacent questions, it indicates that the interviewee's answer quality meets expectations. Questions with higher usage rates and better suited to the target question dimension are selected through a priority threshold. This allows for both in-depth assessment of that dimension and improved assessment efficiency by relying on high-priority questions. Therefore, questions are generated based on the priority threshold. When the accuracy of adjacent questions is less than the preset accuracy of adjacent questions, it indicates that the interviewee's answer quality does not meet expectations. Further analysis of answer stability is needed through the fluctuation dimension, and the information load index is used to determine whether the amount of information assessed is reasonable. This allows for targeted adjustments to the direction of subsequent questions. Therefore, questions are generated based on the fluctuation dimension and the information load index.

[0089] Specifically, it is generated based on the fluctuation dimension and the information load index, including:

[0090] For reference test questions to be generated that have a fluctuation dimension greater than or equal to a preset fluctuation dimension or an information load index greater than or equal to a preset information load index, they are generated based on the evaluation threshold.

[0091] For reference test questions to be generated that have a fluctuation dimension smaller than the preset fluctuation dimension and an information load index smaller than the preset information load index, the generation of the test questions is stopped based on the proximity error rate or dynamically generated based on the learning curve reference value.

[0092] Specifically, for a single reference question to be generated, interview questions that were generated earlier than the reference question to be generated and are adjacent to the reference question to be generated are recorded as neighboring questions.

[0093] The fluctuation dimension is the standard deviation of the cosine similarity between the text vectors corresponding to the answers of the interviewees to the neighboring questions in the historical records of interviewees who meet the user's needs and whose interview results are qualified, and the text vectors corresponding to the standard answers.

[0094] Information load index = number of keywords appearing in adjacent questions / average number of keywords corresponding to each interview question in the interview question bank;

[0095] The values ​​of the preset fluctuation dimension and the preset information load index can be determined by the user according to the actual application scenario. The smaller the values ​​of the preset fluctuation dimension and the preset information load index, the greater the demand generated by the user according to the evaluation threshold. A method for determining the values ​​of the preset fluctuation dimension and the preset information load index is provided. The historical records generated by the user according to the evaluation threshold are detected, and the average value of the fluctuation dimension and the average value of the information load index corresponding to the historical records that can meet the user's demand are respectively recorded as the preset fluctuation dimension and the preset information load index.

[0096] When generating reference questions based on evaluation thresholds, for a single reference question to be generated, the second question with the highest evaluation threshold is used as the reference question for that reference question to be generated.

[0097] The evaluation threshold for a single second question = item fit / average item fit for all second questions + (1 - information load index / average information load index for all second questions).

[0098] The test item fit of a single second question = difficulty similarity / average difficulty similarity of all second questions + question relevance / average question relevance of all second questions;

[0099] The question relevance of a single second question is the cosine similarity between the text vectors corresponding to that second question and the neighboring questions;

[0100] The difficulty similarity of a single second question = 1 / (the absolute value of the difference between the difficulty coefficients of the second question and the adjacent questions + 1); the difficulty coefficient of a single interview question = 1 - (the number of interviewees in the history whose text vectors corresponding to their answers to the interview question and the text vectors corresponding to the standard answer have a cosine similarity greater than 0.6 / the number of interviewees in the history who generated the interview question).

[0101] Understandably, the accuracy of adjacent questions effectively reflects the degree to which the interviewee's answers to adjacent questions match the standard answers. When the accuracy of adjacent questions is greater than or equal to the preset accuracy of adjacent questions, it indicates that the interviewee's answers to adjacent questions meet expectations. By selecting questions with higher usage rates and better suitability to the target question dimension through a priority threshold, we can both continue the in-depth assessment of that dimension and improve the assessment efficiency by relying on high-priority questions. Therefore, the questions are generated based on the priority threshold. When the accuracy of adjacent questions is less than the preset accuracy of adjacent questions, it indicates that the interviewee's answers to adjacent questions do not meet expectations. It is necessary to further analyze the stability of the answers through the fluctuation dimension and judge whether the amount of information assessed is reasonable through the information load index, so as to adjust the direction of subsequent questions in a targeted manner. Therefore, the questions are generated based on the fluctuation dimension and the information load index.

[0102] The fluctuation dimension and information load index effectively reflect the stability of answers to adjacent test questions and the amount of information examined by adjacent test questions. When the fluctuation dimension is greater than or equal to the preset fluctuation dimension or the information load index is greater than or equal to the preset information load index, in order to avoid unstable answers interfering with the accuracy of the assessment or excessive information affecting the true level of the interviewee, it is necessary to optimize the assessment effect by accurately selecting the most suitable test questions. Therefore, it is generated according to the assessment threshold.

[0103] When the fluctuation dimension is less than the preset fluctuation dimension and the information load index is less than the preset information load index, it indicates that the assessment of adjacent test questions is reliable and effective. At this time, it is necessary to further refine the assessment strategy by combining the overall adjacent error rate and the ability improvement trend. Therefore, it is determined to stop generating based on the adjacent error rate or dynamically generate based on the learning curve reference value.

[0104] Specifically, for reference test questions to be generated that have a neighbor error rate lower than a preset neighbor error rate, they are dynamically generated based on the learning curve reference value;

[0105] When dynamically generated based on the learning curve reference value

[0106] For reference test questions to be generated whose learning curve reference value is greater than or equal to the preset learning curve reference value, reference test questions are generated based on the intensity range of the first preset test point.

[0107] For reference questions to be generated where the learning curve reference value is less than the preset learning curve reference value, reference questions are generated based on the intensity range of the second preset examination point.

[0108] Specifically, when generating reference test questions based on the first preset range of test point intensity, for a single reference test question to be generated, any second test question within the first preset range of test point intensity is selected as the reference test question for that reference test question to be generated.

[0109] When generating reference questions based on the second preset range of test point intensity, for a single reference question to be generated, any second question within the second preset range of test point intensity is selected as the reference question for that reference question to be generated.

[0110] If the nearest neighbor error rate is greater than or equal to the preset nearest neighbor error rate, then the generation of reference questions will stop. It should be noted that although the number of reference questions to be generated for each question dimension is n, when the nearest neighbor error rate is greater than or equal to the preset nearest neighbor error rate, reference questions will no longer be generated even if n has not been reached.

[0111] The nearest neighbor error rate is calculated as follows: the number of target generated questions whose cosine similarity between the text vector corresponding to the answer and the text vector corresponding to the standard answer is less than or equal to 0.6 / the total number of target generated questions. The preset nearest neighbor error rate can be determined by the user based on the actual application scenario. The greater the user's requirement for the accuracy of the test question's error tolerance, the smaller the preset nearest neighbor error rate will be. One preset nearest neighbor error rate is provided, which is 0.7.

[0112] The target generated questions whose cosine similarity between the text vector corresponding to the answer and the text vector corresponding to the standard answer is less than or equal to 0.6 are recorded as erroneous questions; the intensity of the test point corresponding to a single second question = the difficulty coefficient corresponding to the second question - the average of the difficulty coefficients corresponding to all erroneous questions;

[0113] After sorting the generated target questions by difficulty coefficient from smallest to largest, a curve is plotted with the difficulty coefficient as the horizontal axis and the cosine similarity between the text vector corresponding to the answer and the text vector corresponding to the standard answer as the vertical axis. It can be understood that each generated target question corresponds to a coordinate point. The curve between two adjacent coordinate points in the coordinate graph is recorded as a curve segment. The learning curve reference value = the total number of curve segments with positive slope / the number of curve segments.

[0114] The user can determine the preset learning curve reference value according to the actual application scenario. The greater the user's need to improve the accuracy of the learning ability assessment, the smaller the preset learning curve reference value should be. One preset learning curve reference value is provided, which is 0.68.

[0115] All values ​​within the intensity range of the first preset test point are greater than or equal to the intensity of the first preset test point;

[0116] The values ​​within the intensity range of the second preset observation point are all less than the intensity of the first preset observation point and greater than the intensity of the second preset observation point; the intensity of the first preset observation point is greater than the intensity of the second preset observation point.

[0117] The values ​​of the first preset assessment point intensity and the second preset assessment point intensity can be determined by the user according to the actual application scenario. The greater the user's need to improve the accuracy of the learning ability assessment, the greater the values ​​of the first preset assessment point intensity and the second preset assessment point intensity. One possible value for the first preset assessment point intensity and the second preset assessment point intensity is 0.4 and 0.1.

[0118] Understandably, the nearest neighbor error rate (NBER) effectively reflects the overall quality of the interviewee's responses and the degree to which their abilities meet the standards in the generated questions for the current assessment dimension. When the NBER is less than the preset NBER, it indicates that the interviewee's overall response quality in that dimension is satisfactory, their core abilities do not show significant weaknesses, and there is still room for further exploration of their ability level and potential for improvement. Therefore, the NBER is dynamically generated based on the learning curve reference value. When the NBER is greater than or equal to the preset NBER, it indicates that the interviewee's overall response quality in that dimension is substandard, their core abilities are significantly insufficient, and continuing to generate questions can no longer effectively improve the assessment accuracy and is unlikely to change the core conclusion that their abilities are not up to standard. Therefore, the generation of reference questions is stopped.

[0119] Specifically, for reference questions to be generated where the analysis status shows that the quantity reference value is less than the preset quantity reference value or the span coefficient is less than the preset span coefficient, reference questions are directly generated based on the knowledge span value.

[0120] When generating reference questions directly based on knowledge span values, for a single reference question to be generated, any second question with a knowledge span value greater than the preset knowledge span value is selected as the reference question for that reference question to be generated.

[0121] Knowledge span value = 1 / (1 + test question fit).

[0122] The value of the preset knowledge span can be determined by the user according to the actual application scenario. The greater the user's need for improving the accuracy of the coverage of interview content, the larger the value of the preset knowledge span. A method for determining the value of the preset knowledge span is provided, which is the average value of the knowledge span corresponding to each reference question in the history of the reference questions that can meet the user's needs and directly generate reference questions based on the knowledge span value.

[0123] Specifically, template matching is performed when the generated status is that the request difficulty value is greater than or equal to the preset request difficulty value and the maximum matching coefficient is greater than or equal to the preset maximum matching coefficient.

[0124] The difficulty value of the request is determined based on the number of interviewees of a certain type and the estimated generation time.

[0125] Specifically, the generation status includes a first generation status and a second generation status. The first generation status is when the request difficulty value is greater than or equal to the preset request difficulty value and the maximum matching coefficient is greater than or equal to the preset maximum matching coefficient. The second generation status is when the request difficulty value is less than the preset request difficulty value or the maximum matching coefficient is less than the preset maximum matching coefficient.

[0126] The present invention has a continuously cyclical monitoring cycle. At the end of each monitoring cycle, the generated status is determined. The duration of the monitoring cycle can be set according to the user's needs. The greater the user's need for monitoring accuracy, the shorter the duration of the monitoring cycle. One possible value for the monitoring cycle is 5 minutes.

[0127] Request difficulty value = Number of interviewees of one type / Average number of interviewees of one type corresponding to the historical records that can meet the user's needs + Estimated generation time / Average estimated generation time corresponding to the historical records that can meet the user's needs;

[0128] The number of interviewees in category 1 is the total number of interviewees in category 1 at the beginning of the current monitoring period;

[0129] Users can obtain the time required to generate interview questions for each type of interviewee in the historical generation process. The average time required to generate interview questions for each type of interviewee in the historical generation process is recorded as the sub-generation time. The estimated generation time = the number of type of interviewees at the beginning of the current monitoring period × the sub-generation time. The unit of the estimated generation time is ms.

[0130] For a single Category II interviewee, the maximum matching coefficient is the maximum value of the matching coefficients between the resume information of the Category II interviewee and each candidate template. The matching coefficient between the resume information of the Category II interviewee and a single candidate template = resume matching degree × resume weight coefficient + job matching degree × job weight coefficient; both the resume weight coefficient and the job weight coefficient are 0.5.

[0131] Resume matching score = Number of keywords in the resume that have a keyword similarity greater than the preset keyword similarity with any keyword in the template to be selected / Number of keywords in the resume; Job matching score = Number of keywords in the target job that have a keyword similarity greater than the preset keyword similarity with any keyword in the template to be selected / Number of keywords in the target job.

[0132] In template matching, for a single second-category interviewee, the interview questions from the candidate templates with the highest matching coefficient are selected as the interview questions for that second-category interviewee;

[0133] Specifically, when the generated status is that the request difficulty value is less than the preset request difficulty value or the maximum matching coefficient is less than the preset maximum matching coefficient, template combination is performed.

[0134] Specifically, in the template combination, for a single second-type interviewee, templates are selected for each question dimension. When selecting a template for a single question dimension, the interview questions corresponding to that question dimension in the candidate template with the highest dimension matching degree are used as the interview questions generated for that question dimension for the second-type interviewee. Then, template selection continues for question dimensions that have not been selected until corresponding interview questions are generated for each question dimension.

[0135] The dimension matching degree corresponding to a single question dimension is the cosine similarity between the text vector corresponding to the target job information and the text vectors of all interview questions corresponding to that question dimension in the template to be selected;

[0136] Understandably, the request difficulty value and maximum matching coefficient effectively reflect the task pressure of generating interview questions and the accuracy of matching with the templates to be selected. When the generation status is that the request difficulty value is greater than or equal to the preset request difficulty value and the maximum matching coefficient is greater than or equal to the preset maximum matching coefficient, it indicates that the current task pressure of generating interview questions is relatively high, and there are ready-made templates that are highly compatible with the interviewees and target job information. Template matching can balance generation efficiency and adaptability, so template matching is performed. When the generation status is that the request difficulty value is less than the preset request difficulty value or the maximum matching coefficient is less than the preset maximum matching coefficient, it indicates that the current task pressure of generating interview questions is relatively low, or there are no ready-made templates that are highly compatible with the interviewees and target job information. Template combination is needed to achieve accurate coverage of various dimensions, so template combination is performed.

[0137] Specifically, for interviewees whose interview evaluation coefficient is less than the preset interview evaluation coefficient, the interview is deemed unqualified.

[0138] For interviewees whose interview evaluation coefficient is greater than or equal to the preset interview evaluation coefficient, the interview is deemed qualified.

[0139] The interview evaluation coefficient for a class of interviewees = the first adjustment coefficient × the average cosine similarity between the text vectors corresponding to the answers to each interview question and the text vectors corresponding to the standard answers. The first adjustment coefficient is 1.1.

[0140] The interview evaluation coefficient for the second type of interviewees is calculated as follows: the second adjustment coefficient is multiplied by the average cosine similarity between the text vectors of the answers to each interview question and the text vectors of the corresponding standard answers. The second adjustment coefficient is 0.9.

[0141] The preset interview evaluation coefficient can be determined by the user based on the actual application scenario. The greater the user's need to improve the accuracy of the interview evaluation, the larger the preset interview evaluation coefficient will be. A method for setting the preset interview evaluation coefficient is provided, which detects the historical records of qualified interviews and records the average value of the interview evaluation coefficients corresponding to the historical records that meet the user's needs as the preset interview evaluation coefficient.

[0142] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A comprehensive interview evaluation method based on reinforcement learning, characterized in that, include: Obtain information about the target job position and the resumes of several interviewees for the corresponding job position; The categories of interviewees are determined based on the degree of information matching and reference values ​​for working hours, and then sequential matching or direct matching is performed based on the categories of interviewees. In sequential matching, the target questions for each question dimension are determined based on the question matching degree. The analysis status of the reference questions to be generated is determined based on the quantity reference value and the span coefficient. Based on the analysis status, it is determined whether to generate reference questions through proximity analysis or directly generate reference questions based on the knowledge span value. In the proximity analysis generation, generation is determined based on the accuracy of neighboring test items, either by a priority threshold or by the fluctuation dimension and information load index. In direct matching, the generation status is determined based on the request difficulty value and the maximum matching coefficient, and template matching or template combination is performed based on the generation status to generate test questions; The interview pass / fail status is determined based on the interview evaluation coefficient. For interviewees whose information matching degree is greater than or equal to the preset information matching degree or whose working time reference value is greater than or equal to the preset working time reference value, sequential matching is performed; For reference test questions to be generated whose analysis status is that the quantity reference value is greater than or equal to the preset quantity reference value and the span coefficient is greater than or equal to the preset span coefficient, a proximity analysis is performed to generate them; Proximity analysis is in progress. For reference questions to be generated where the accuracy of neighboring questions is greater than or equal to the preset accuracy of neighboring questions, they are generated according to a priority threshold. For reference test questions to be generated where the accuracy of neighboring test questions is lower than the preset accuracy of neighboring test questions, they are generated based on the fluctuation dimension and information load index. It is generated based on the fluctuation dimension and information load index, including: For reference test questions to be generated that have a fluctuation dimension greater than or equal to a preset fluctuation dimension or an information load index greater than or equal to a preset information load index, they are generated based on the evaluation threshold. For reference test questions to be generated that have a fluctuation dimension smaller than the preset fluctuation dimension and an information load index smaller than the preset information load index, the generation is stopped based on the proximity error rate or dynamically generated based on the learning curve reference value. For reference test questions to be generated with a neighbor error rate lower than the preset neighbor error rate, they are dynamically generated based on the learning curve reference value; When dynamically generated based on the learning curve reference value For reference test questions to be generated whose learning curve reference value is greater than or equal to the preset learning curve reference value, reference test questions are generated based on the intensity range of the first preset test point. For reference questions to be generated where the learning curve reference value is less than the preset learning curve reference value, reference questions are generated based on the intensity range of the second preset examination point.

2. The comprehensive interview evaluation method based on reinforcement learning according to claim 1, characterized in that, For two types of interviewees, whose information matching degree is less than the preset information matching degree and whose working time reference value is less than the preset working time reference value, direct matching is performed; The categories of interviewees include Category I interviewees and Category II interviewees.

3. The comprehensive interview evaluation method based on reinforcement learning according to claim 1, characterized in that, For reference questions to be generated where the analysis status shows that the quantity reference value is less than the preset quantity reference value or the span coefficient is less than the preset span coefficient, reference questions are directly generated based on the knowledge span value.

4. The comprehensive interview evaluation method based on reinforcement learning according to claim 1, characterized in that, When the generation status is that the request difficulty value is greater than or equal to the preset request difficulty value and the maximum matching coefficient is greater than or equal to the preset maximum matching coefficient, template matching is performed. The difficulty value of the request is determined based on the number of interviewees of a certain type and the estimated generation time.

5. The comprehensive interview evaluation method based on reinforcement learning according to claim 4, characterized in that, When the generated status is that the request difficulty value is less than the preset request difficulty value or the maximum matching coefficient is less than the preset maximum matching coefficient, template combination is performed.

6. The comprehensive interview evaluation method based on reinforcement learning according to claim 1, characterized in that, For interviewees whose interview evaluation coefficient is less than the preset interview evaluation coefficient, the interview is deemed unqualified. For interviewees whose interview evaluation coefficient is greater than or equal to the preset interview evaluation coefficient, the interview is deemed qualified.

Images