Scoring and Follow-up Method, Device and Electronic Device Based on Interview Question Answers

Abstract

The present invention provides a method, apparatus and electronic device for scoring and follow-up questioning based on interview question answers, belonging to the technical field of artificial intelligence. In this method, multiple similarity measurement algorithms are integrated to achieve a multi-angle quantitative evaluation of the answers of candidates. Additionally, a large model scoring link based on deep learning is creatively added to make up for the depth of language understanding that cannot be reached by pure mathematical calculations. This dual-track scoring architecture relies on an accurate mathematical model to ensure the objectivity and consistency of scoring on the one hand; on the other hand, it uses a large model with high semantic parsing ability to enhance the subjective understanding and interpretability of scoring, improving the accuracy and comprehensiveness of evaluation. The above process is automatically implemented with high efficiency, that is, the method of the present invention can quickly, accurately and comprehensively evaluate the answers of candidates. In addition, the scoring reasons and follow-up questions given by the large model provide valuable information for exploring the knowledge mastery of candidates.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and in particular, to a method, device, electronic device, and storage medium for scoring and follow-up questioning based on the answers to interview questions. Background Art

[0002] In today's highly competitive and rapidly digitizing job market, enterprises are facing unprecedented challenges and opportunities. On the one hand, with the development of the economy and the continuous growth of the talent market, the number of resumes received by enterprises is increasing exponentially. On the other hand, in order to break through the limitations of geographical space, various remote online recruitment methods have emerged rapidly. In this process, how to quickly, accurately, and real-time evaluate the answers to the interview questions of candidates has become an urgent problem to be solved, and it is a severe challenge for enterprises.

[0003] Currently, traditional evaluation methods rely too much on the subjective judgment and experience of interviewers, are extremely vulnerable to personal biases, and are inefficient and costly; or use fixed rules or shallow machine learning models to evaluate through keyword matching statistics and simple machine learning model features. These methods are only suitable for processing structured standard data. Although they are simple to implement and have low costs, when it comes to question answers that require a deep understanding of context semantics or logical structures, they will lead to serious biases in the evaluation of interview results and cannot guarantee their accuracy and fairness.

[0004] In summary, traditional methods for evaluating question answers cannot ensure both efficiency and the accuracy of evaluation. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, device, and electronic device for scoring and follow-up questioning based on the answers to interview questions, so as to alleviate the technical problem that traditional methods for evaluating question answers cannot ensure both efficiency and the accuracy of evaluation.

[0006] In a first aspect, an embodiment of the present invention provides a method for scoring and follow-up questioning based on the answers to interview questions, including:

[0007] Obtain the candidate's answer to the current interview question, and perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question to obtain a candidate answer vector and a reference answer vector, where the reference answer includes at least one of the following: simple answer, standard answer, extended answer;

[0008] Calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate answer vector and the reference answer vector respectively;

[0009] Perform a weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity to obtain the formula total score of the candidate's answer;

[0010] Based on the current interview question, the candidate's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of tokens, construct a scoring prompt, and input the scoring prompt into the large model to output the large model total score and the scoring reason of the candidate's answer;

[0011] Perform a weighted average calculation on the formula total score and the large model total score to obtain the final score of the candidate's answer, and evaluate the candidate's answer according to the final score and the preset score threshold, and then perform subsequent processing according to the evaluation result.

[0012] Further, obtain the candidate's answer to the current interview question, including:

[0013] After the candidate logs in to the AI interview software, the AI interview software obtains the corresponding target question from the question bank according to the candidate's basic information, the applied position, and the position level, and presents the current interview question in the target question in text and online voice broadcast;

[0014] Obtain the audio answer of the candidate answering the current interview question, and convert the audio answer into text to obtain the candidate's answer.

[0015] Further, perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question, including:

[0016] Input the candidate's answer and the preset reference answer to the current interview question into an embedding model for vectorization processing to obtain the candidate's answer vector and the reference answer vector.

[0017] Further, calculate the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity between the candidate's answer vector and the reference answer vector, including:

[0018] According to the cosine similarity calculation formula Calculate the cosine similarity between the candidate's answer vector and the reference answer vector, where represents the cosine similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, represents the dimension of the candidate's answer vector or the reference answer vector;

[0019] According to the Euclidean distance similarity calculation formula Calculate the Euclidean distance similarity between the candidate's answer vector and the reference answer vector, where represents the Euclidean distance similarity represents the i-th dimension of the reference answer vector represents the i-th dimension of the candidate's answer vector represents the dimension of the candidate's answer vector or the reference answer vector;

[0020] According to the Manhattan distance similarity calculation formula Calculate the Manhattan distance similarity between the candidate's answer vector and the reference answer vector, where represents the Manhattan distance similarity represents the i-th dimension of the reference answer vector represents the i-th dimension of the candidate's answer vector represents the dimension of the candidate's answer vector or the reference answer vector.

[0021] Furthermore, if the cosine similarity includes: the cosine similarity between the candidate's answer and the simple answer, the cosine similarity between the candidate's answer and the standard answer, the cosine similarity between the candidate's answer and the extended answer, the Euclidean distance similarity includes: the Euclidean distance similarity between the candidate's answer and the simple answer, the Euclidean distance similarity between the candidate's answer and the standard answer, the Euclidean distance similarity between the candidate's answer and the extended answer, the Manhattan distance similarity includes: the Manhattan distance similarity between the candidate's answer and the simple answer, the Manhattan distance similarity between the candidate's answer and the standard answer, the Manhattan distance similarity between the candidate's answer and the extended answer, perform a weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity, including:

[0022] According to the formula total score calculation formula 、 、 、 Calculate the formula total score of the candidate's answer, where represents the formula total score of the candidate's answer represents the total score of the cosine similarity represents the total score of the Euclidean distance similarity represents the total score of the Manhattan distance similarity represents the weight corresponding to the total score of the cosine similarity represents the weight corresponding to the total score of the Euclidean distance similarity represents the weight corresponding to the total score of the Manhattan distance similarity represents the cosine similarity between the applicant's answer and the simple answer, represents the cosine similarity between the applicant's answer and the standard answer, represents the cosine similarity between the applicant's answer and the extended answer, represents the Euclidean distance similarity between the applicant's answer and the simple answer, represents the Euclidean distance similarity between the applicant's answer and the standard answer, represents the Euclidean distance similarity between the applicant's answer and the extended answer, represents the Manhattan distance similarity between the applicant's answer and the simple answer, represents the Manhattan distance similarity between the applicant's answer and the standard answer, represents the Manhattan distance similarity between the applicant's answer and the extended answer, represents the weight corresponding to the simple answer, represents the weight corresponding to the standard answer, represents the weight corresponding to the extended answer.

[0023] Furthermore, a weighted average calculation is performed on the formula total score and the large model total score, including:

[0024] According to the final score calculation formula Calculate the final score of the applicant's answer, where, represents the final score of the applicant's answer, represents the formula total score, represents the large model total score, represents the weight corresponding to the formula total score, represents the weight corresponding to the large model total score.

[0025] Furthermore, the applicant's answer is evaluated according to the final score and a preset score threshold, and then subsequent processing is performed according to the evaluation result, including:

[0026] If the final score is not less than the preset score threshold, it is determined that the applicant's answer is complete and valid, and proceed to the next interview question;

[0027] If the final score is less than the preset score threshold and the number of follow-up questions has not been exceeded, then based on the current interview question, the candidate's answer, the reference answer, the sampling temperature parameter, and the description of the follow-up question to be generated, a prompt for generating a follow-up question is constructed and input into the large model to output a follow-up question, so that the candidate can answer the follow-up question. Among them, the follow-up question is generated for the content lacking in the candidate's answer;

[0028] If the final score is less than the preset score threshold, the number of follow-up questions has been exceeded, or the candidate's answer is determined to be an invalid answer, then the next interview question is entered.

[0029] In a second aspect, an embodiment of the present invention further provides a scoring and follow-up device for interview question answers, including:

[0030] A vectorization processing unit, configured to obtain the candidate's answer to the current interview question, and perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question to obtain a candidate's answer vector and a reference answer vector, where the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer;

[0031] A calculation unit, configured to calculate the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity between the candidate's answer vector and the reference answer vector respectively;

[0032] A weighted average calculation unit, configured to perform a weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity to obtain the formula total score of the candidate's answer;

[0033] A scoring prompt word construction unit, configured to construct a scoring prompt word based on the current interview question, the candidate's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of tokens, and input the scoring prompt word into the large model to output the large model total score and the scoring reason of the candidate's answer;

[0034] A weighted average calculation and evaluation unit, configured to perform a weighted average calculation on the formula total score and the large model total score to obtain the final score of the candidate's answer, and evaluate the candidate's answer according to the final score and the preset score threshold, and then perform subsequent processing according to the evaluation result.

[0035] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the method according to any one of the first aspects are implemented.

[0036] In an embodiment of the present invention, a method for scoring and questioning based on answers to interview questions is provided, comprising: obtaining an answer from an applicant to a current interview question, and vectorizing the applicant's answer and a preset reference answer to the current interview question to obtain an applicant's answer vector and a reference answer vector, wherein the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; calculating cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the applicant's answer vector and the reference answer vector respectively; performing a weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain a formula total score for the applicant's answer; constructing a scoring prompt based on the current interview question, the applicant's answer, the reference answer, a scoring rule, a scoring standard, a scoring reason, a sampling temperature parameter, and a maximum number of tokens, inputting the scoring prompt into a large model, and outputting a large model total score and a scoring reason for the applicant's answer; performing a weighted average calculation on the formula total score and the large model total score to obtain a final score for the applicant's answer, evaluating the applicant's answer based on the final score and a preset score threshold, and performing subsequent processing based on the evaluation result. From the above description, it can be seen that the method for scoring and questioning based on the answers to interview questions of the present invention integrates a variety of similarity measurement algorithms to achieve a multi-angle quantitative evaluation of the applicant's answers, and also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that cannot be reached by pure mathematical calculations. This dual-track scoring architecture, on the one hand, relies on a precise mathematical model to ensure the objectivity and consistency of the scoring; on the other hand, it uses a large model with a high degree of semantic parsing ability to enhance the subjective understanding and explanatory power of the scoring, greatly improving the accuracy and comprehensiveness of the evaluation. In addition, the above process is automated, so it is highly efficient. That is, the method for scoring and questioning based on the answers to interview questions of the present invention can quickly, accurately and comprehensively evaluate the applicant's answers. In addition, the scoring reasons and follow-up questions given by the large model provide valuable information for exploring the applicant's knowledge mastery, alleviating the technical problem that the traditional question answer evaluation method cannot ensure efficiency while taking into account the accuracy of the evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A flowchart of a method for scoring and questioning based on answers to interview questions provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of a device for scoring and questioning based on answers to interview questions provided by an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] Traditional question answer evaluation methods cannot ensure both efficiency and accuracy.

[0043] Based on this, the method of scoring and asking questions based on the answers to interview questions of the present invention integrates a variety of similarity measurement algorithms to achieve a multi-angle quantitative evaluation of the applicant's answers, and also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that cannot be reached by pure mathematical calculations. This dual-track scoring architecture, on the one hand, relies on a precise mathematical model to ensure the objectivity and consistency of the scoring; on the other hand, it uses a large model with a high degree of semantic parsing ability to enhance the subjective understanding and explanatory power of the scoring, greatly improving the accuracy and comprehensiveness of the evaluation. In addition, the above process is automated, so it is highly efficient, that is, the method of scoring and asking questions based on the answers to interview questions of the present invention can quickly, accurately and comprehensively evaluate the applicant's answers. In addition, the scoring reasons and follow-up questions given by the large model provide valuable information for exploring the applicant's knowledge mastery.

[0044] To facilitate understanding of this embodiment, a method for scoring and questioning based on answers to interview questions disclosed in an embodiment of the present invention is first introduced in detail.

[0045] Example 1:

[0046] According to an embodiment of the present invention, an embodiment of a method for scoring and questioning based on answers to interview questions is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0047] Figure 1It is a flowchart of a method for scoring and follow-up questions based on the answers to interview questions according to an embodiment of the present invention. As Figure 1 shown, the method includes the following steps:

[0048] Step S102, obtain the applicant's answer to the current interview question, and perform vectorization processing on the applicant's answer and the preset reference answer to the current interview question to obtain the applicant's answer vector and the reference answer vector, where the reference answer includes at least one of the following: simple answer, standard answer, extended answer;

[0049] In the embodiment of the present invention, 5 roles are involved, namely the applicant, the AI interview software (i.e., the AI interview digital human), the AI back-end service, the AI large model, and the embedding model. Specifically, the AI back-end service obtains the applicant's answer to the current interview question, and then inputs the applicant's answer and the preset reference answer to the current interview question into the embedding model for vectorization processing to obtain the applicant's answer vector and the reference answer vector.

[0050] The reference answers are divided into three levels: primary, intermediate, and advanced, corresponding to different weight scores respectively. The reference answer corresponding to the primary level is called the simple answer, the reference answer corresponding to the intermediate level is called the standard answer, and the reference answer corresponding to the advanced level is called the extended answer.

[0051] Different reference answers correspond to different depths of knowledge. Simple answer: The concept is simple and clear, easy to understand, focusing on intuitive understanding without complex technical details; Standard answer: Focus on technical accuracy, avoid being too basic, be able to explain the core principle of the concept, and provide ideas for practical applications; Extended answer: Use a large number of technical terms and professional descriptions, focus on details and principles, be precise and complete in description, be able to relate to multiple technical fields, and demonstrate the applicant's breadth and depth of knowledge.

[0052] Simple answer: Suitable for basic positions or non-technical scenarios, with the focus on allowing people to quickly understand the concept.

[0053] Standard answer: Suitable for intermediate positions or conventional technical assessments, examining both basic knowledge and practical applications.

[0054] Extended answer: Suitable for senior positions, evaluating theoretical depth and technical professionalism, emphasizing details and innovative thinking.

[0055] For example

[0056] Question: Please briefly describe the differences between machine learning and deep learning?

[0057] Simple Answer (Primary Answer): Machine learning is a method of training models using data, including supervised learning, unsupervised learning, and reinforcement learning. Deep learning is a branch of machine learning that uses neural networks to automatically extract features and is suitable for processing complex data types such as images and speech. Weight: 0.2.

[0058] Standard Answer (Intermediate Answer): Machine learning is a method of training models through data to complete specific tasks, divided into supervised learning, unsupervised learning, and reinforcement learning, and usually relies on manually designed feature engineering. Deep learning is a subfield of machine learning, based on a multi-layer neural network architecture, capable of automatically extracting complex features from data, especially suitable for unstructured data (such as images, audio, and text), without the need for manual feature extraction. Weight: 0.3.

[0059] Extended Answer (Advanced Answer): Machine learning is a collection of algorithms that build predictive models by learning from labeled or unlabeled data. It relies on traditional statistical methods such as linear regression, support vector machines, and random forests, and usually requires human participation in feature engineering to optimize model performance. Deep learning is a subset of machine learning, with the core being multi-layer neural networks, which can automatically learn feature representations through large-scale data and computing resources. The essence of deep learning lies in simulating the working mechanism of the human brain, capable of extracting hierarchical features of data in a high-dimensional space, thus achieving significant performance improvements in complex tasks (such as image recognition, natural language processing, etc.), but its training process has high requirements for computing resources and data scale. Weight: 0.5.

[0060] The above preset reference answers can be one of the simple answer, standard answer, and extended answer, or two, or even three, mainly depending on the administrator's customized configuration. If the reference answer is the simple answer, the obtained reference answer vector is the simple answer vector; if the reference answer is the simple answer and the standard answer, the obtained reference answer vector is the simple answer vector and the standard answer vector.

[0061] Step S104: Calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate's answer vector and the reference answer vector respectively.

[0062] Specifically, the calculation processes of the above cosine similarity, Euclidean distance similarity, and Manhattan distance similarity are executed by the AI back-end service.

[0063] Step S106: Perform a weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain the formula total score of the candidate's answer.

[0064] Specifically, the process of calculating the formula total score is executed by the AI back-end service.

[0065] Step S108: Construct a scoring prompt based on the current interview question, the candidate's answer, the reference answer, the scoring rules, the scoring criteria, the reasons for scoring, the sampling temperature parameter, and the maximum number of tokens, and input the scoring prompt into the large model to output the total score of the candidate's answer and the reasons for scoring obtained from the large model.

[0066] Specifically, the AI back-end service constructs a scoring prompt (such as, current interview question / candidate's answer / reference answer / scoring rules / invalid or avoidance criteria / scoring criteria / reasons for scoring / sampling temperature parameter / maximum number of tokens) based on information such as the current interview question, the candidate's answer, the reference answer, the scoring rules of 0-100 (for example, 100 points for a completely correct answer, 80 points for an 80% match, 60 points for a 60% match, and so on), the invalid or avoidance criteria (do not understand, not clear, not familiar, do not know, not good at, not proficient in, cannot, do not understand, have not been exposed to, have not been involved in, have not mastered, have not learned, skip, etc.), the scoring criteria, the reasons for scoring, the sampling temperature parameter, and the maximum number of tokens, and then feeds it to the large model to output the total score of the candidate's answer and the reasons for scoring obtained from the large model.

[0067] The AI large model reasons, evaluates, and scores according to the scoring prompt and gives the reasons for scoring. The AI back-end service parses the results returned by the large model and extracts the score and the reasons for scoring.

[0068]

[0069] Step S110: Perform a weighted average calculation on the formula total score and the large model total score to obtain the final score of the candidate's answer, evaluate the candidate's answer according to the final score and the preset score threshold, and then perform subsequent processing according to the evaluation result.

[0070] In an embodiment of the present invention, a method for scoring and questioning based on answers to interview questions is provided, comprising: obtaining an answer from an applicant to a current interview question, and vectorizing the applicant's answer and a preset reference answer to the current interview question to obtain an applicant's answer vector and a reference answer vector, wherein the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; calculating cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the applicant's answer vector and the reference answer vector respectively; performing a weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain a formula total score for the applicant's answer; constructing a scoring prompt based on the current interview question, the applicant's answer, the reference answer, a scoring rule, a scoring standard, a scoring reason, a sampling temperature parameter, and a maximum number of tokens, inputting the scoring prompt into a large model, and outputting a large model total score and a scoring reason for the applicant's answer; performing a weighted average calculation on the formula total score and the large model total score to obtain a final score for the applicant's answer, evaluating the applicant's answer based on the final score and a preset score threshold, and performing subsequent processing based on the evaluation result. From the above description, it can be seen that the method of scoring and questioning based on the answers to interview questions of the present invention integrates a variety of similarity measurement algorithms to achieve a multi-angle quantitative evaluation of the applicant's answers, and also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that cannot be reached by pure mathematical calculations. This dual-track scoring architecture, on the one hand, relies on a precise mathematical model to ensure the objectivity and consistency of the scoring; on the other hand, it uses a large model with a high degree of semantic parsing ability to enhance the subjective understanding and explanatory power of the scoring, greatly improving the accuracy and comprehensiveness of the evaluation. In addition, the above process is automated, so it is highly efficient, that is, the method of scoring and questioning based on the answers to interview questions of the present invention can quickly, accurately and comprehensively evaluate the applicant's answers. In addition, the scoring reasons and follow-up questions given by the large model provide valuable information for exploring the applicant's knowledge mastery, alleviating the technical problem that the traditional question answer evaluation method cannot ensure efficiency while taking into account the accuracy of the evaluation.

[0071] The above content briefly introduces the method of scoring and questioning based on answers to interview questions of the present invention. The specific contents involved are described in detail below.

[0072] In an optional embodiment of the present invention, obtaining the candidate's answer to the current interview question specifically includes the following steps:

[0073] (1) When the applicant logs in to the AI interview software, the AI interview software obtains the corresponding target test questions from the test question bank based on the applicant's basic information, job position, and job level, and presents the current interview questions in the target test questions in text and online voice broadcast;

[0074] When implemented, after the applicant logs in to the AI interview software, the AI interview digital human will retrieve target questions of corresponding difficulty from the question bank according to the applicant's basic information, the interviewed position, the required job level, etc., and then present the current interview question in the target question in text and play it via online voice broadcast.

[0075] (2) Obtain the audio answer of the applicant to the current interview question, and convert the audio answer into text to obtain the applicant's answer.

[0076] Specifically, the applicant starts answering the question according to the current interview question. To improve the answering efficiency, the applicant will answer the question through the microphone.

[0077] For example, the applicant answers: "Machine learning is a method of training a model using data to solve specific tasks. It includes supervised learning, unsupervised learning, and reinforcement learning, and usually requires manual design of feature extraction methods to improve the model's performance. Deep learning is a branch of machine learning that automatically extracts complex features through a multi-layer neural network architecture, and is particularly suitable for processing unstructured data without manual intervention in feature engineering."

[0078] The AI interview digital human records the audio in real time and transmits the audio answer to the AI backend service. The AI backend service converts the audio answer of the applicant's answer into text to obtain the applicant's answer.

[0079] In an optional embodiment of the present invention, the applicant's answer and the preset reference answer to the current interview question are vectorized, which specifically includes the following steps:

[0080] Input the applicant's answer and the preset reference answer to the current interview question into the embedding model for vectorization to obtain the applicant's answer vector and the reference answer vector.

[0081] Specifically, the AI backend service will retrieve the preset reference answer to the current interview question from the question bank, and perform vectorization (Embedding) with the applicant's answer input into the embedding model, and convert the reference answer and the applicant's answer into high-dimensional vectors A (reference answer vector) and B (applicant's answer vector).

[0082] The role of the embedding model (trained based on BERT) is to map high-dimensional sparse data (such as words or sentences) into a low-dimensional dense vector space to capture the semantic similarity between data. Here, BERT (Bidirectional Encoder Representations from Transformers), which shows excellent performance in natural language processing tasks, is adopted. It is a bidirectional encoder based on the Transformer architecture. By introducing masked language modeling (MLM) and next sentence prediction (NSP), combined with the self-attention mechanism, it conducts unsupervised pre-training on a large amount of corpus to understand the context relationship of words in a sentence, thereby capturing deeper semantic information in the text.

[0083] During pre-training, BERT randomly selects some tokens in the input text sequence for masking and lets the model predict these masked words. This can enable the model to learn to infer the missing information based on the surrounding context. Specifically, BERT randomly selects 15% of the words in the input text sequence for masking. Then, 80% of them are replaced with a special [MASK] token, 10% remain unchanged, and the other 10% are randomly replaced with other words. That is, assuming we have an input sequence containing 100 words, according to BERT's masking strategy, 15% of the words will be selected for masking: 15% of 100 words is 15 words, which will be selected for masking. Among these 15 selected words: 80% of the words (i.e., 12 out of 15 words) will be replaced with the [MASK] token. 10% of the words (i.e., 1 out of 15 words) will remain unchanged. 10% of the words (i.e., 2 out of 15 words) will be randomly replaced with other words. This is to enable the model to learn not only the context information of the masked words but also the context information of the unmasked words, thereby improving the generalization ability of the model.

[0084] The vectors generated by Embedding have multiple dimensions, such as 512 dimensions, 768 dimensions, 1024 dimensions. Each dimension is a floating-point number (float), which is used to represent the mapping value of words in the high-dimensional space, reflecting the relative position and semantic relationship characteristics of words in the multi-dimensional space. The higher the precision of the floating-point value, the more subtle differences and complex relationships in semantics can be captured.

[0085] Dimensions greater than three cannot be intuitively represented by graphs. However, whether it is high-dimensional or low-dimensional, their calculation methods are the same. Therefore, the following takes the visualizable three-dimensional vectors as an example for calculation and explanation.

[0086] Example:

[0087] Hypothesis: The standard answer in the reference answer is represented as (i.e., the standard answer vector) after being vectorized:

[0088]

[0089] The candidate's answer vector is represented as:

[0090]

[0091] In an alternative embodiment of the present invention, the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate's answer vector and the reference answer vector are calculated respectively, and the specific steps are as follows:

[0092] (1) Calculate the cosine similarity between the candidate's answer vector and the reference answer vector according to the cosine similarity calculation formula where, represents the cosine similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, represents the dimension of the candidate's answer vector or the reference answer vector;

[0093] (2) Calculate the Euclidean distance similarity between the candidate's answer vector and the reference answer vector according to the Euclidean distance similarity calculation formula where, represents the Euclidean distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, represents the dimension of the candidate's answer vector or the reference answer vector;

[0094] (3) Calculate the Manhattan distance similarity between the candidate's answer vector and the reference answer vector according to the Manhattan distance similarity calculation formula where, represents the Manhattan distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, represents the dimension of the candidate's answer vector or the reference answer vector.

[0095] Specifically, the AI back-end service calculates the cosine similarity between the candidate's answer vector and the reference answer vector:

[0096] Calculate: ;

[0097]

[0098]

[0099] Calculation: ;

[0100]

[0101] Calculation: ;

[0102]

[0103]

[0104] The AI back-end service calculates the Euclidean distance similarity between the applicant's answer vector and the reference answer vector:

[0105] Sub-item calculation:

[0106]

[0107] Accumulate and take the square root to make the result consistent with the original scale:

[0108]

[0109]

[0110] The AI back-end service calculates the Manhattan distance similarity between the applicant's answer vector and the reference answer vector:

[0111] Sub-item calculation:

[0112]

[0113] Accumulate the scores:

[0114]

[0115]

[0116] If the similarity is 1, it means that the applicant's answer is exactly the same as the standard answer; if it is 0, it means that the applicant's answer has no relation to the standard answer. The closer it is to 1, the more similar it is, and the closer it is to 0, the less similar it is.

[0117] In an alternative embodiment of the present invention, if the cosine similarity includes: the cosine similarity between the candidate's answer and the simple answer, the cosine similarity between the candidate's answer and the standard answer, the cosine similarity between the candidate's answer and the extended answer, the Euclidean distance similarity includes: the Euclidean distance similarity between the candidate's answer and the simple answer, the Euclidean distance similarity between the candidate's answer and the standard answer, the Euclidean distance similarity between the candidate's answer and the extended answer, and the Manhattan distance similarity includes: the Manhattan distance similarity between the candidate's answer and the simple answer, the Manhattan distance similarity between the candidate's answer and the standard answer, the Manhattan distance similarity between the candidate's answer and the extended answer, a weighted average calculation is performed on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity, which specifically includes the following steps:

[0118] According to the formula total score calculation formula , , , Calculate the total formula score of the candidate's answer, where represents the total formula score of the candidate's answer, represents the total cosine similarity score, represents the total Euclidean distance similarity score, represents the total Manhattan distance similarity score, represents the weight corresponding to the total cosine similarity score, represents the weight corresponding to the total Euclidean distance similarity score, represents the weight corresponding to the total Manhattan distance similarity score, represents the cosine similarity between the candidate's answer and the simple answer, represents the cosine similarity between the candidate's answer and the standard answer, represents the cosine similarity between the candidate's answer and the extended answer, represents the Euclidean distance similarity between the candidate's answer and the simple answer, represents the Euclidean distance similarity between the candidate's answer and the standard answer, represents the Euclidean distance similarity between the candidate's answer and the extended answer, represents the Manhattan distance similarity between the candidate's answer and the simple answer, represents the Manhattan distance similarity between the candidate's answer and the standard answer, represents the Manhattan distance similarity between the candidate's answer and the extended answer, represents the weight corresponding to the simple answer, represents the weight corresponding to the standard answer, represents the weight corresponding to the extended answer.

[0119] Specifically, for the AI backend service, the weighted average method is used to calculate the total formula score. The formula is: Total formula score = (Cosine similarity total score × 0.6 weight) + (Euclidean distance similarity total score × 0.2 weight) + (Manhattan distance similarity total score × 0.2 weight), and then it is converted into a score between 0 and 100. Among them, the weight value can be customized according to the actual scenario.

[0120] As in the above example: The cosine similarity is the cosine similarity between the candidate's answer and the standard answer. Then the cosine similarity total score = the cosine similarity between the candidate's answer and the standard answer (0.9745). The Euclidean distance similarity is the Euclidean distance similarity between the candidate's answer and the standard answer. Then the Euclidean distance similarity total score = the Euclidean distance similarity between the candidate's answer and the standard answer (0.76). The Manhattan distance similarity is the Manhattan distance similarity between the candidate's answer and the standard answer. Then the Manhattan distance similarity total score = the Manhattan distance similarity between the candidate's answer and the standard answer (0.69).

[0121] Total formula score =

[0122] The above calculation process is only for the calculation between the standard answer and the candidate's answer. In the actual application process, parallel tensor matrix calculations will also be performed with the simple answer (primary answer) and the extended answer (advanced answer). The calculation process is the same, but the data is different. Whether to calculate this depends on whether the interviewer configures it.

[0123] The reason for this design is that intermediate-level candidates may also answer more difficult answers, so they should have a higher weighted score.

[0124] Therefore, the calculation formula for the total formula score is:

[0125] 、 、 、 where, represents the total formula score of the candidate's answer, represents the cosine similarity total score, represents the Euclidean distance similarity total score, represents the Manhattan distance similarity total score, represents the weight corresponding to the cosine similarity total score, represents the weight corresponding to the Euclidean distance similarity total score, represents the weight corresponding to the Manhattan distance similarity total score, represents the cosine similarity between the candidate's answer and the simple answer, represents the cosine similarity between the candidate's answer and the standard answer, represents the cosine similarity between the candidate's answer and the extended answer, Indicates the Euclidean distance similarity between the candidate's answer and the simple answer. Indicates the Euclidean distance similarity between the candidate's answer and the standard answer. Indicates the Euclidean distance similarity between the candidate's answer and the extended answer. Indicates the Manhattan distance similarity between the candidate's answer and the simple answer. Indicates the Manhattan distance similarity between the candidate's answer and the standard answer. Indicates the Manhattan distance similarity between the candidate's answer and the extended answer. Indicates the weight corresponding to the simple answer. Indicates the weight corresponding to the standard answer. Indicates the weight corresponding to the extended answer.

[0126] In an alternative embodiment of the present invention, a weighted average calculation is performed on the formula total score and the large model total score, which specifically includes the following steps:

[0127] According to the final score calculation formula Calculate the final score of the candidate's answer, where Indicates the final score of the candidate's answer. Indicates the formula total score. Indicates the large model total score. Indicates the weight corresponding to the formula total score. Indicates the weight corresponding to the large model total score.

[0128] Specifically, the AI back-end service calculates the final score of the candidate's answer again. The weighted average method formula is: Final score = (Explicit algorithm logic score (i.e., formula total score) * 0.6 weight) + (Large model score (i.e., large model total score) * 0.4 weight), where the weight value can be configured as needed.

[0129] As the above example, =

[0130] Converted to a percentage, and finally the final score is 82.

[0131] By combining the explicit algorithms of cosine similarity, Euclidean distance similarity, and Manhattan distance similarity with the large model score, the final score is calculated, and the final score is 82 points.

[0132] The explicit algorithm quantifies the matching degree between the candidate's answer and the standard answer through mathematical formulas, providing interpretability and transparency; while the large model scoring utilizes the context understanding ability of deep learning to comprehensively evaluate semantics, logic, and language quality for intelligent assessment, making up for the deficiency of the explicit algorithm in understanding complex semantic relationships and further supplementing the explicit algorithm. The combination of these two methods not only ensures the scientificity, objectivity, and fairness of the scoring process but also enhances the flexibility and semantic adaptation ability of the assessment, achieving a balance between high precision and high interpretability.

[0133] In an alternative embodiment of the present invention, the candidate's answer is evaluated based on the final score and a preset score threshold, and subsequent processing is then performed according to the evaluation result, specifically including the following steps:

[0134] (1) If the final score is not less than the preset score threshold, it is determined that the candidate's answer is complete and valid, and the next interview question is entered.

[0135] (2) If the final score is less than the preset score threshold and the number of follow-up questions has not been exceeded, a prompt word for generating a follow-up question is constructed based on the current interview question, the candidate's answer, the reference answer, the sampling temperature parameter, and the description of the content lacking in the candidate's answer, and the prompt word for generating a follow-up question is input into the large model to output the follow-up question, and then the candidate is asked to answer the follow-up question, where the follow-up question is generated for the content lacking in the candidate's answer.

[0136] (3) If the final score is less than the preset score threshold and the number of follow-up questions has been exceeded, or the candidate's answer is determined to be an invalid answer, the next interview question is entered.

[0137] Specifically, the AI back-end service determines whether the final score is not less than the specified preset score threshold. If the final score is not less than the preset score threshold, it is determined that the answer is complete and valid, and the next interview question is entered; if the final score is less than the preset score threshold, it is determined that the answer content is lacking. Under this condition, it is continued to determine whether the current interview question has reached the maximum number of follow-up questions. If the maximum number of follow-up questions has not been reached, a prompt word for generating a follow-up question will be constructed again (the candidate's answer, the current interview question, the reference answer, the sampling temperature parameter, the description of generating a follow-up question: Please compare the candidate's answer with the standard answer / the answer generated by the large model according to the current interview question to evaluate the deficiencies in the candidate's answer content, and generate a new follow-up question based on the deficiencies). The large model will regenerate a follow-up question for the lacking content of the answer, then convert the follow-up question from text to speech, and finally the AI interview digital human will broadcast the speech to the candidate, enabling the candidate to answer the follow-up question, and the processes from step S102 to step S110 above will be carried out again to evaluate the follow-up question. If the final score is less than the preset score threshold and the maximum number of follow-up questions for the current interview question is reached, the next interview question will continue to be broadcast. There is also a situation where the candidate's score is 0, the answer is wrong, the answer is invalid, or the candidate refuses to answer. After reaching the maximum number of follow-up questions for the current interview question, this current interview question will also be skipped and the next interview question will be broadcast. The above all processes of evaluating each interview question in a loop are carried out until the candidate has answered all the interview questions in the target test questions, and finally it ends.

[0138] In the method of the present invention, first, the candidate logs in to the AI interview software and answers the interview questions. Then, the candidate's answer and the reference answers of three difficulty levels of simple, standard, and advanced preset in advance are vectorized, and the corresponding score values are calculated through the Euclidean distance similarity, Manhattan distance similarity, and cosine similarity. Each similarity algorithm is assigned different weights according to its characteristics, and each reference answer level is also assigned different weights to ensure that the scoring result can comprehensively and all-round consider the candidate's answer. Then, the explicit formula total score is calculated using the multi-layer weighted average method.

[0139] Next, the interview question, the reference answers of three difficulty levels, and the candidate's answer, etc. are constructed into a scoring prompt word and input into the pre-trained large model together to obtain a more intelligent large model total score. Then, the explicit formula total score calculated and the large model total score are weighted and synthesized in proportion to obtain the final score. If the final score reaches the preset score threshold, it is considered that the answer is valid and the next interview question can be entered; otherwise, if the final score is lower than the preset score threshold and the number of follow-up questions has not been exceeded, the candidate is allowed to further clarify the answer content of the follow-up question; and when the final score is lower than the preset score threshold and the number of follow-up questions has been exceeded, or it is determined as an invalid answer, there will be no further follow-up questions, and the next interview question will be directly entered.

[0140] Different from the scoring method purely based on mathematical formulas, the large model scoring not only provides the total score, but also gives detailed reasons for the scores, points out the advantages and disadvantages in the candidates' answers, and generates follow-up questions for incomplete or lacking parts to understand the candidates' knowledge level more deeply.

[0141] Adopting this dual-track scoring mechanism not only achieves multi-dimensional accurate evaluation, but also enhances the transparency and rationality of scoring, making the entire evaluation process more scientific and fair.

[0142] The purpose of the present invention is to develop an efficient, fair and intelligent automatic scoring and evaluation mechanism, which aims to meet the need for rapid and accurate evaluation of candidates' answers to questions.

[0143] This solution has innovations in many aspects:

[0144] First, it combines multiple similarity algorithm scores. Instead of simply using a single algorithm to measure the similarity of answers, by assigning different algorithm weights, it can not only reflect the absolute difference but also the semantic relevance, and can comprehensively consider the similarity degree of answers in different dimensions, making the scoring results more comprehensive and scientific. Euclidean distance focuses on the actual distance between two points in the vector space, Manhattan distance pays more attention to the sum of distances along the coordinate axes, and cosine similarity focuses on the consistency of vector directions. By comprehensively using these three algorithms and making full use of the advantages of each algorithm, the relationship between answers can be analyzed from multiple angles.

[0145] Second, it has a hierarchical weighting mechanism for answers of different difficulty levels. Simple answers, standard answers and extended answers contribute differently to evaluating the quality of candidates' answers. By reasonably allocating weights, the evaluation can more accurately reflect the actual level of candidates' answers. Simple answers may cover the basic points, and their higher weights can ensure key consideration of the mastery of basic content; while advanced answers reflect a deeper understanding.

[0146] Third, it introduces a large model based on deep learning for scoring. The large model has powerful context language understanding and analysis capabilities, can evaluate answers from multiple complex levels such as semantics, logic, and integrity, and makes up for the deficiencies of simply relying on similarity algorithms in semantic understanding and logical coherence analysis. The scoring reasons and follow-up questions given by the large model provide valuable information for further exploring the candidates' knowledge mastery.

[0147] In summary, the main purpose of this invention is to establish an efficient, fair, scientific, and user-friendly scoring and questioning mechanism, providing strong support and reliable assurance for remote interviews in enterprises, significantly reducing labor costs and improving interview efficiency. Its innovation lies in its breakthrough in the limitations of traditional scoring methods. By integrating multiple advanced technologies, it establishes a flexible, intelligent, and efficient scoring framework, bringing new changes to the field of automated assessment.

[0148] Key points of the present invention:

[0149] The key protection lies in the construction of a unique dual-track hybrid scoring system. This system not only integrates multiple distance measurement algorithms to achieve a multi-faceted quantitative assessment of applicants' answers, but also innovatively incorporates a large-scale model scoring process based on deep learning, thereby complementing the depth of language understanding that pure mathematical calculations cannot achieve. This dual-track scoring architecture relies on precise mathematical models to ensure the objectivity and consistency of scoring; on the other hand, it leverages a large-scale model with advanced semantic parsing capabilities to enhance the subjective understanding and interpretability of the scores. To maintain scoring transparency, the large-scale model also provides specific justifications for each score (i.e., score rationale), allowing applicants to clearly understand their strengths and weaknesses. Furthermore, considering the varying value of different types of reference answers, the present invention has designed a tiered weighting mechanism, assigning customized weights based on the difficulty of the reference answers, ensuring that the scoring results objectively and accurately reflect the applicant's actual level of skill. Furthermore, the present invention emphasizes the rebalancing of the outputs of different similarity algorithms, optimizing the composition of the final score by setting their respective weights, thus preventing the possibility of a single algorithm dominating the scoring process.

[0150] This invention also places special emphasis on interactivity and dynamic adjustment within the scoring process. By feeding the large model with prompts constructed from interview questions, simple answers, standard answers, advanced answers, and candidate answers, it not only generates an overall score but also identifies potential issues in the candidate's answers. For areas that are lacking or incomplete, the large model automatically generates follow-up questions to further verify the candidate's depth of understanding and mastery of the knowledge. This mechanism not only increases scoring flexibility but also provides candidates with more opportunities to demonstrate their abilities.

[0151] Generally speaking, the key protection points of the present invention are reflected in the following aspects: First, a multi-dimensional quantitative evaluation model is established. By combining different distance measurement algorithms and a hierarchical weighting mechanism, the comprehensiveness and accuracy of the scoring results are ensured; Second, a large model with powerful semantic parsing capabilities is introduced to improve the subjective understanding and interpretability of scoring; Third, the transparency and interactivity of the scoring process are emphasized. By providing detailed scoring reasons and generating follow-up questions, the candidate experience is enhanced; Fourth, reasonable preset score thresholds and follow-up question limits are set to ensure the scientificity and fairness of the evaluation results.

[0152] Embodiment 2:

[0153] The embodiment of the present invention also provides a scoring follow-up device based on the answers to interview questions. This scoring follow-up device based on the answers to interview questions is mainly used to execute the scoring follow-up method based on the answers to interview questions provided in Embodiment 1 of the present invention. The following is a specific introduction to the scoring follow-up device based on the answers to interview questions provided by the embodiment of the present invention.

[0154] Figure 2 is a schematic diagram of a scoring follow-up device based on the answers to interview questions according to an embodiment of the present invention, as Figure 2 shown. The device mainly includes: a vectorization processing unit 10, a calculation unit 20, a weighted average calculation unit 30, a scoring prompt word construction unit 40, and a weighted average calculation and evaluation unit 50, where:

[0155] The vectorization processing unit is used to obtain the candidate's answer to the current interview question, and perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question to obtain a candidate's answer vector and a reference answer vector. Among them, the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer;

[0156] The calculation unit is used to calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate's answer vector and the reference answer vector respectively;

[0157] The weighted average calculation unit is used to perform weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain the formula total score of the candidate's answer;

[0158] The scoring prompt word construction unit is used to construct a scoring prompt word based on the current interview question, the candidate's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of Tokens, and input the scoring prompt word into the large model to output the large model total score and the scoring reason of the candidate's answer;

[0159] The weighted average calculation and evaluation unit is used to perform weighted average calculation on the total score of the formula and the total score of the large model to obtain the final score of the applicant's answer, and evaluate the applicant's answer based on the final score and the preset score threshold, and then perform subsequent processing based on the evaluation result.

[0160] In an embodiment of the present invention, a device for scoring and questioning based on answers to interview questions is provided, comprising: obtaining an answer from an applicant to a current interview question, and vectorizing the applicant's answer and a preset reference answer to the current interview question to obtain an applicant's answer vector and a reference answer vector, wherein the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; calculating cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the applicant's answer vector and the reference answer vector respectively; performing a weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain a formula total score for the applicant's answer; constructing a scoring prompt based on the current interview question, the applicant's answer, the reference answer, a scoring rule, a scoring standard, a scoring reason, a sampling temperature parameter, and a maximum number of tokens, inputting the scoring prompt into a large model, and outputting a large model total score and a scoring reason for the applicant's answer; performing a weighted average calculation on the formula total score and the large model total score to obtain a final score for the applicant's answer, evaluating the applicant's answer based on the final score and a preset score threshold, and performing subsequent processing based on the evaluation result. From the above description, it can be seen that the scoring and questioning device based on the answers to interview questions of the present invention integrates a variety of similarity measurement algorithms to achieve a multi-angle quantitative evaluation of the applicant's answers, and also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that cannot be reached by pure mathematical calculations. This dual-track scoring architecture, on the one hand, relies on a precise mathematical model to ensure the objectivity and consistency of the scoring; on the other hand, it uses a large model with a high degree of semantic parsing ability to enhance the subjective understanding and explanatory power of the scoring, greatly improving the accuracy and comprehensiveness of the evaluation. In addition, the above process is automated, so it is highly efficient. That is, the scoring and questioning method based on the answers to interview questions of the present invention can quickly, accurately and comprehensively evaluate the applicant's answers. In addition, the scoring reasons and follow-up questions given by the large model provide valuable information for exploring the applicant's knowledge mastery, alleviating the technical problem that the traditional question answer evaluation method cannot ensure efficiency while taking into account the accuracy of the evaluation.

[0161] Optionally, the vectorized processing unit is also used for: when the applicant logs in to the AI interview software, the AI interview software obtains the corresponding target test questions from the test question bank based on the applicant's basic information, applied position, and position level, and presents the current interview questions in the target test questions in text and online voice broadcast; obtains the audio answers of the applicant to the current interview questions, and converts the audio answers into text to obtain the applicant's answers.

[0162] Optionally, the vectorization processing unit is further used to: input the applicant's answer and the preset reference answer to the current interview question into the embedding model for vectorization processing to obtain the applicant's answer vector and the reference answer vector.

[0163] Optionally, the calculation unit is further configured to: calculate the formula according to the cosine similarity Calculate the cosine similarity between the candidate's answer vector and the reference answer vector, where represents the cosine similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, Indicates the dimension of the candidate's answer vector or reference answer vector; similarity calculation formula based on Euclidean distance Calculate the Euclidean distance similarity between the candidate's answer vector and the reference answer vector, where represents the Euclidean distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, Represents the dimension of the candidate's answer vector or reference answer vector; based on the Manhattan distance similarity calculation formula Calculate the Manhattan distance similarity between the candidate's answer vector and the reference answer vector, where represents the Manhattan distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, Represents the dimension of the candidate's answer vector or reference answer vector.

[0164] Optionally, if the cosine similarity includes: the cosine similarity between the candidate's answer and the simple answer, the cosine similarity between the candidate's answer and the standard answer, and the cosine similarity between the candidate's answer and the extended answer; the Euclidean distance similarity includes: the Euclidean distance similarity between the candidate's answer and the simple answer, the Euclidean distance similarity between the candidate's answer and the standard answer, and the Euclidean distance similarity between the candidate's answer and the extended answer; and the Manhattan distance similarity includes: the Manhattan distance similarity between the candidate's answer and the simple answer, the Manhattan distance similarity between the candidate's answer and the standard answer, and the Manhattan distance similarity between the candidate's answer and the extended answer, the weighted average calculation unit is further configured to: calculate the total score according to the formula 、 、 、 Calculate the total score of the candidate's answer using the formula, where The total score of the formula representing the candidate's answer, represents the total cosine similarity score, Represents the total Euclidean distance similarity score, Represents the total Manhattan distance similarity score, Represents the weight corresponding to the total cosine similarity score, Represents the weight corresponding to the total Euclidean distance similarity score, Represents the weight corresponding to the total Manhattan distance similarity score, Represents the cosine similarity between the candidate's answer and the simple answer, Represents the cosine similarity between the candidate's answer and the standard answer, Represents the cosine similarity between the candidate's answer and the extended answer, Represents the Euclidean distance similarity between the candidate's answer and the simple answer, Represents the Euclidean distance similarity between the candidate's answer and the standard answer, Represents the Euclidean distance similarity between the candidate's answer and the extended answer, Represents the Manhattan distance similarity between the candidate's answer and the simple answer, Represents the Manhattan distance similarity between the candidate's answer and the standard answer, Represents the Manhattan distance similarity between the candidate's answer and the extended answer, Represents the weight corresponding to the simple answer, Represents the weight corresponding to the standard answer, Represents the weight corresponding to the extended answer.

[0165] Optionally, the weighted average calculation and evaluation unit is also used to: calculate the final score of the candidate's answer according to the final score calculation formula where, Represents the final score of the candidate's answer, Represents the total formula score, Represents the total large model score, Represents the weight corresponding to the total formula score, Represents the weight corresponding to the total large model score.

[0166] Optionally, the weighted average calculation and evaluation unit is also used to: if the final score is not less than the preset score threshold, determine that the candidate's answer is complete and valid, and proceed to the next interview question; if the final score is less than the preset score threshold and the number of follow-up questions has not been exceeded, then based on the current interview question, the candidate's answer, the reference answer, the sampling temperature parameter, construct a prompt for generating a follow-up question to describe the generated follow-up question, and input the generated follow-up question prompt into the large model to output the follow-up question, so that the candidate can answer the follow-up question, where the follow-up question is generated for the content lacking in the candidate's answer; if the final score is less than the preset score threshold and the number of follow-up questions has been exceeded, or the candidate's answer is determined to be an invalid answer, then proceed to the next interview question.

[0167] The device provided by the embodiment of the present invention has the same implementation principle and technical effects as those of the foregoing method embodiment. For the sake of brief description, for the parts not mentioned in the device embodiment, reference may be made to the corresponding content in the foregoing method embodiment.

[0168] As Figure 3 As shown in the figure, an electronic device 600 provided by an embodiment of the present application includes: a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device runs, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the method for scoring and following up according to the interview question answers as described above.

[0169] Specifically, the foregoing memory 602 and processor 601 can be general-purpose memory and processor, which are not specifically limited herein. When the processor 601 runs the computer program stored in the memory 602, it can execute the method for scoring and following up according to the interview question answers as described above.

[0170] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the foregoing method can be completed by the integrated logic circuit in the hardware of the processor 601 or the instructions in software form. The foregoing processor 601 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by the hardware decoding processor, or executed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and combines its hardware to complete the steps of the foregoing method.

[0171] The scoring follow-up device according to the interview question answers provided by the embodiments of the present application may be specific hardware on the device, or software or firmware installed on the device, etc. For the device provided by the embodiments of the present application, its implementation principle and the technical effects produced are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the foregoing-described systems, devices, and units can all refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0172] In the embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some communication interfaces. The indirect coupling or communication connection of the devices or units may be in an electrical, mechanical, or other form.

[0173] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of the blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

[0176] If the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method for scoring and follow-up questions according to the interview question answers described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM for short), random access memories (RAM for short), magnetic disks, or optical discs that can store program codes.

[0177] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0178] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of the present application, used to illustrate the technical solutions of the present application, rather than limiting them. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art within the technical scope disclosed in the present application can still modify the technical solutions described in the foregoing embodiments, or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application. All should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for scoring and follow-up questioning based on interview question answers, characterized in that, Including: Obtain the candidate's answer to the current interview question, and perform vectorization processing on the candidate's answer and the preset reference answer of the current interview question to obtain a candidate answer vector and a reference answer vector, where the reference answer includes at least one of the following: simple answer, standard answer, extended answer; Calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate answer vector and the reference answer vector respectively; Perform weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity to obtain the formula total score of the candidate answer; Construct a scoring prompt word based on the current interview question, the candidate answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of tokens, and input the scoring prompt word into the large model to output the large model total score and the scoring reason of the candidate answer; Perform weighted average calculation on the formula total score and the large model total score to obtain the final score of the candidate answer, and evaluate the candidate answer according to the final score and the preset score threshold, and then perform subsequent processing according to the evaluation result; Among them, if the cosine similarity includes: the cosine similarity between the candidate answer and the simple answer, the cosine similarity between the candidate answer and the standard answer, the cosine similarity between the candidate answer and the extended answer, the Euclidean distance similarity includes: the Euclidean distance similarity between the candidate answer and the simple answer, the Euclidean distance similarity between the candidate answer and the standard answer, the Euclidean distance similarity between the candidate answer and the extended answer, the Manhattan distance similarity includes: the Manhattan distance similarity between the candidate answer and the simple answer, the Manhattan distance similarity between the candidate answer and the standard answer, the Manhattan distance similarity between the candidate answer and the extended answer, performing weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity includes: According to the formula, calculate the total score formula , , , Calculate the total score of the formula for the applicant's answer, where represents the total score of the formula for the applicant's answer, represents the total score of cosine similarity, represents the total score of Euclidean distance similarity, represents the total score of Manhattan distance similarity, represents the weight corresponding to the total score of cosine similarity, represents the weight corresponding to the total score of Euclidean distance similarity, represents the weight corresponding to the total score of Manhattan distance similarity, represents the cosine similarity between the applicant's answer and the simple answer, represents the cosine similarity between the applicant's answer and the standard answer, represents the cosine similarity between the applicant's answer and the extended answer, represents the Euclidean distance similarity between the applicant's answer and the simple answer, represents the Euclidean distance similarity between the applicant's answer and the standard answer, represents the Euclidean distance similarity between the applicant's answer and the extended answer, represents the Manhattan distance similarity between the applicant's answer and the simple answer, represents the Manhattan distance similarity between the applicant's answer and the standard answer, represents the Manhattan distance similarity between the applicant's answer and the extended answer, represents the weight corresponding to the simple answer, represents the weight corresponding to the standard answer, represents the weight corresponding to the extended answer; Among them, evaluating the candidate answer according to the final score and the preset score threshold, and then performing subsequent processing according to the evaluation result includes: If the final score is not less than the preset score threshold, it is determined that the candidate answer is complete and valid, and proceed to the next interview question; If the final score is less than the preset score threshold and the number of follow-up questions has not been exceeded, based on the current interview question, the candidate answer, the reference answer, the sampling temperature parameter, construct a generation follow-up question prompt word for the description of the follow-up question, and input the generation follow-up question prompt word into the large model to output the follow-up question, and then enable the candidate to answer the follow-up question, where the follow-up question is generated for the content lacking in the candidate's answer; If the final score is less than the preset score threshold and the number of follow-up questions has been exceeded, or the candidate answer is determined to be an invalid answer, then proceed to the next interview question.

2. The method according to claim 1, wherein Obtain the candidate's answer to the current interview question, including: After the candidate logs in to the AI interview software, the AI interview software obtains the corresponding target questions from the question bank according to the candidate's basic information, applied position, and position level, and presents the current interview question in the target questions in text and plays it online as voice. Obtain the audio answer of the candidate answering the current interview question, and convert the audio answer into text to obtain the candidate's answer.

3. The method according to claim 1, characterized in that, Perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question, including: Input the candidate's answer and the preset reference answer to the current interview question into the embedding model for vectorization processing to obtain the candidate's answer vector and the reference answer vector.

4. The method according to claim 1, wherein Calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate's answer vector and the reference answer vector respectively, including: According to the cosine similarity calculation formula Calculate the cosine similarity between the candidate's answer vector and the reference answer vector, where represents the cosine similarity represents the i-th dimension of the reference answer vector represents the i-th dimension of the candidate's answer vector represents the dimension of the candidate's answer vector or the reference answer vector; According to the Euclidean distance similarity calculation formula Calculate the Euclidean distance similarity between the candidate's answer vector and the reference answer vector, where represents the Euclidean distance similarity represents the i-th dimension of the reference answer vector represents the i-th dimension of the candidate's answer vector represents the dimension of the candidate's answer vector or the reference answer vector; According to the Manhattan distance similarity calculation formula Calculate the Manhattan distance similarity between the candidate answer vector and the reference answer vector, where represents the Manhattan distance similarity represents the i-th dimension of the reference answer vector represents the i-th dimension of the candidate answer vector represents the dimension of the candidate answer vector or the reference answer vector 5. The method according to claim 1, wherein Perform weighted average calculation on the formula total score and the large model total score, including: Calculation formula for the final score Calculate the final score of the applicant's answer, where represents the final score of the applicant's answer, represents the total score of the formula, represents the total score of the large model, represents the weight corresponding to the total score of the formula, represents the weight corresponding to the total score of the large model.

6. A scoring follow-up device based on interview question answers, characterized in that, Include: The vectorization processing unit is used to obtain the candidate's answer to the current interview question, and perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question to obtain the candidate's answer vector and the reference answer vector, where the reference answer includes at least one of the following: simple answer, standard answer, extended answer; The calculation unit is used to calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate's answer vector and the reference answer vector respectively; The weighted average calculation unit is used to perform weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity to obtain the formula total score of the candidate's answer; The scoring prompt word construction unit is used to construct a scoring prompt word based on the current interview question, the candidate's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum Token number, and input the scoring prompt word into the large model to output the large model total score and the scoring reason of the candidate's answer; The weighted average calculation and evaluation unit is used to perform weighted average calculation on the formula total score and the large model total score to obtain the final score of the candidate's answer, and evaluate the candidate's answer according to the final score and the preset score threshold, and then perform subsequent processing according to the evaluation result; Among them, if the cosine similarity includes: the cosine similarity between the candidate's answer and the simple answer, the cosine similarity between the candidate's answer and the standard answer, and the cosine similarity between the candidate's answer and the extended answer, the Euclidean distance similarity includes: the Euclidean distance similarity between the candidate's answer and the simple answer, the Euclidean distance similarity between the candidate's answer and the standard answer, and the Euclidean distance similarity between the candidate's answer and the extended answer, and the Manhattan distance similarity includes: the Manhattan distance similarity between the candidate's answer and the simple answer, the Manhattan distance similarity between the candidate's answer and the standard answer, and the Manhattan distance similarity between the candidate's answer and the extended answer, the weighted average calculation unit is further configured to: according to the formula total score calculation formula , , , calculate the total score of the candidate's answer, where represents the total score of the formula of the candidate's answer, represents the total score of the cosine similarity, represents the total score of the Euclidean distance similarity, represents the total score of the Manhattan distance similarity, represents the weight corresponding to the total score of the cosine similarity, represents the weight corresponding to the total score of the Euclidean distance similarity, represents the weight corresponding to the total score of the Manhattan distance similarity, represents the cosine similarity between the candidate's answer and the simple answer, represents the cosine similarity between the candidate's answer and the standard answer, represents the cosine similarity between the candidate's answer and the extended answer, represents the Euclidean distance similarity between the candidate's answer and the simple answer, represents the Euclidean distance similarity between the candidate's answer and the standard answer, represents the Euclidean distance similarity between the candidate's answer and the extended answer, represents the Manhattan distance similarity between the candidate's answer and the simple answer, represents the Manhattan distance similarity between the candidate's answer and the standard answer, represents the Manhattan distance similarity between the candidate's answer and the extended answer, represents the weight corresponding to the simple answer, represents the weight corresponding to the standard answer, represents the weight corresponding to the extended answer; Among them, the weighted average calculation and evaluation unit is further configured to: if the final score is not less than the preset score threshold, determine that the candidate's answer is complete and valid, and proceed to the next interview question; if the final score is less than the preset score threshold and the number of follow-up questions has not been exceeded, based on the current interview question, the candidate's answer, the reference answer, the sampling temperature parameter, construct a prompt word for generating a follow-up question by describing the content lacking in the candidate's answer, and input the prompt word for generating a follow-up question into the large model to output a follow-up question, so that the candidate can answer the follow-up question, where the follow-up question is generated for the content lacking in the candidate's answer; if the final score is less than the preset score threshold, and the number of follow-up questions has been exceeded, or the candidate's answer is determined to be an invalid answer, then proceed to the next interview question.

7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5 above.

Images