Method and device for scoring and questioning according to answers to interview questions and electronic equipment

Abstract

The invention provides a method and device for scoring and questioning according to interview question answers and electronic equipment, and belongs to the technical field of artificial intelligence, in the method, multiple similarity measurement algorithms are integrated to achieve multi-angle quantitative evaluation of applicant answers, a large model scoring link based on deep learning is creatively added, and the evaluation efficiency is improved. Therefore, the language understanding depth which cannot be reached by pure mathematical calculation is compensated. According to the double-track scoring architecture, on one hand, the objectivity and the consistency of scoring are guaranteed by means of an accurate mathematical model; on the other hand, subjective understanding and interpretation of scoring are enhanced by means of a large model with high semantic analysis ability, the accuracy and comprehensiveness of evaluation are improved, the process is automatically achieved, efficiency is high, and the method can rapidly, accurately and comprehensively evaluate the applicant answers. In addition, the scoring reason and the question-asking problem given by the large model provide valuable information for mining the knowledge mastering condition of the applicant.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and in particular, to a method, device, electronic device, and storage medium for scoring and follow-up questioning based on the answers to interview questions. Background Art

[0002] In today's highly competitive and rapidly digitizing job market, enterprises are facing unprecedented challenges and opportunities. On the one hand, with the development of the economy and the continuous growth of the talent market, the number of resumes received by enterprises is increasing exponentially. On the other hand, in order to break through the limitations of geographical space, various remote online recruitment methods have emerged rapidly. In this process, how to quickly, accurately, and real-time evaluate the answers to the interview questions of candidates has become an urgent problem to be solved, and it is a severe challenge facing enterprises.

[0003] Currently, traditional evaluation methods rely too much on the subjective judgment and experience of interviewers, are extremely vulnerable to personal biases, and are inefficient and costly; or use fixed rules or shallow machine learning models to evaluate through keyword matching statistics and simple machine learning model features. These methods are only suitable for processing structured standard data. Although they are simple to implement and have low costs, when it comes to question answers that require a deep understanding of context semantics or logical structures, they will lead to serious biases in the evaluation of interview results and cannot guarantee their accuracy and fairness.

[0004] In summary, traditional methods for evaluating question answers cannot balance efficiency and the accuracy of evaluation. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, device, and electronic device for scoring and follow-up questioning based on the answers to interview questions, so as to alleviate the technical problem that traditional methods for evaluating question answers cannot balance efficiency and the accuracy of evaluation.

[0006] In a first aspect, an embodiment of the present invention provides a method for scoring and follow-up questioning based on the answers to interview questions, including: Obtain the candidate's answer to the current interview question, and perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question to obtain a candidate answer vector and a reference answer vector, where the reference answer includes at least one of the following: simple answer, standard answer, extended answer; Calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate answer vector and the reference answer vector respectively; Perform weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity to obtain the formula total score of the candidate answer; Construct a scoring prompt word based on the current interview question, the candidate's answer, the reference answer, the scoring rules, the scoring criteria, the reasons for scoring, the sampling temperature parameter, and the maximum number of tokens, and input the scoring prompt word into the large model to output the total score of the large model for the candidate's answer and the reasons for scoring; Perform a weighted average calculation on the total formula score and the total large model score to obtain the final score of the candidate's answer, and evaluate the candidate's answer based on the final score and the preset score threshold, and then perform subsequent processing according to the evaluation result.

[0007] Further, obtain the candidate's answer to the current interview question, including: After the candidate logs in to the AI interview software, the AI interview software obtains the corresponding target questions from the question bank according to the candidate's basic information, the applied position, and the position level, and presents the current interview question in the target questions in text and online voice broadcast; Obtain the audio answer of the candidate answering the current interview question, and convert the audio answer into text to obtain the candidate's answer.

[0008] Further, perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question, including: Input the candidate's answer and the preset reference answer to the current interview question into the embedding model for vectorization processing to obtain the candidate's answer vector and the reference answer vector.

[0009] Further, calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate's answer vector and the reference answer vector, including: According to the cosine similarity calculation formula Calculate the cosine similarity between the candidate's answer vector and the reference answer vector, where represents the cosine similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, represents the dimension of the candidate's answer vector or the reference answer vector; According to the Euclidean distance similarity calculation formula Calculate the Euclidean distance similarity between the candidate's answer vector and the reference answer vector, where represents the Euclidean distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, represents the dimension of the candidate answer vector or the reference answer vector; According to the Manhattan distance similarity calculation formula calculate the Manhattan distance similarity between the candidate answer vector and the reference answer vector, where represents the Manhattan distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate answer vector, represents the dimension of the candidate answer vector or the reference answer vector.

[0010] Furthermore, if the cosine similarity includes: the cosine similarity between the candidate answer and the simple answer, the cosine similarity between the candidate answer and the standard answer, the cosine similarity between the candidate answer and the extended answer, the Euclidean distance similarity includes: the Euclidean distance similarity between the candidate answer and the simple answer, the Euclidean distance similarity between the candidate answer and the standard answer, the Euclidean distance similarity between the candidate answer and the extended answer, the Manhattan distance similarity includes: the Manhattan distance similarity between the candidate answer and the simple answer, the Manhattan distance similarity between the candidate answer and the standard answer, the Manhattan distance similarity between the candidate answer and the extended answer, perform a weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity, including: According to the formula total score calculation formula 、 、 、 calculate the formula total score of the candidate answer, where represents the formula total score of the candidate answer, represents the total cosine similarity score, represents the total Euclidean distance similarity score, represents the total Manhattan distance similarity score, represents the weight corresponding to the total cosine similarity score, represents the weight corresponding to the total Euclidean distance similarity score, represents the weight corresponding to the total Manhattan distance similarity score, represents the cosine similarity between the candidate answer and the simple answer, represents the cosine similarity between the candidate answer and the standard answer, represents the cosine similarity between the candidate answer and the extended answer, represents the Euclidean distance similarity between the candidate answer and the simple answer, represents the Euclidean distance similarity between the candidate answer and the standard answer, Indicates the Euclidean distance similarity between the candidate's answer and the extended answer. Indicates the Manhattan distance similarity between the candidate's answer and the simple answer. Indicates the Manhattan distance similarity between the candidate's answer and the standard answer. Indicates the Manhattan distance similarity between the candidate's answer and the extended answer. Indicates the weight corresponding to the simple answer. Indicates the weight corresponding to the standard answer. Indicates the weight corresponding to the extended answer.

[0011] Further, a weighted average calculation is performed on the formula total score and the large model total score, including: According to the final score calculation formula Calculate the final score of the candidate's answer, where Indicates the final score of the candidate's answer. Indicates the formula total score. Indicates the large model total score. Indicates the weight corresponding to the formula total score. Indicates the weight corresponding to the large model total score.

[0012] Further, the candidate's answer is evaluated according to the final score and a preset score threshold, and then subsequent processing is performed according to the evaluation result, including: If the final score is not less than the preset score threshold, it is determined that the candidate's answer is complete and valid, and the next interview question is entered. If the final score is less than the preset score threshold and the number of follow-up questions has not been exceeded, a follow-up question prompt word is constructed based on the current interview question, the candidate's answer, the reference answer, the sampling temperature parameter, and the description of generating the follow-up question, and the generated follow-up question prompt word is input into the large model to output a follow-up question, and then the candidate answers the follow-up question. The follow-up question is generated for the content lacking in the candidate's answer. If the final score is less than the preset score threshold, and the number of follow-up questions has been exceeded, or the candidate's answer is determined to be an invalid answer, then the next interview question is entered.

[0013] In a second aspect, an embodiment of the present invention further provides a scoring and follow-up device for interview question answers, including: A vectorization processing unit, configured to obtain an applicant's answer to the current interview question, and perform vectorization processing on the applicant's answer and a preset reference answer to the current interview question to obtain an applicant's answer vector and a reference answer vector, where the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; A calculation unit, configured to calculate the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity between the applicant's answer vector and the reference answer vector respectively; A weighted average calculation unit, configured to perform a weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity to obtain the formula total score of the applicant's answer; A scoring prompt word construction unit, configured to construct a scoring prompt word based on the current interview question, the applicant's answer, the reference answer, a scoring rule, a scoring standard, a scoring reason, a sampling temperature parameter, and a maximum Token number, and input the scoring prompt word into a large model to output the large model total score and the scoring reason of the applicant's answer; A weighted average calculation and evaluation unit, configured to perform a weighted average calculation on the formula total score and the large model total score to obtain the final score of the applicant's answer, and evaluate the applicant's answer according to the final score and a preset score threshold, and then perform subsequent processing according to the evaluation result.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the steps of the method according to any one of the first aspects are implemented.

[0015] In an embodiment of the present invention, a scoring and questioning method based on answers to interview questions is provided, including: obtaining an answer of an applicant to a current interview question, and vectorizing the applicant's answer and a preset reference answer to the current interview question to obtain an applicant's answer vector and a reference answer vector, wherein the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; respectively calculating the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity of the applicant's answer vector and the reference answer vector; performing weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain a formula total score of the applicant's answer; constructing scoring prompt words based on the current interview question, the applicant's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of tokens, and inputting the scoring prompt words into a large model to output the large model total score and the scoring reason of the applicant's answer; performing weighted average calculation on the formula total score and the large model total score to obtain a final score of the applicant's answer, and evaluating the applicant's answer according to the final score and a preset score threshold, and then performing subsequent processing according to the evaluation result. From the above description, it can be seen that the scoring and questioning method according to the answers to the interview questions of the present invention integrates a variety of similarity measurement algorithms to achieve a multi-angle quantitative evaluation of the applicant's answers, and also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that cannot be touched by pure mathematical calculations. This dual-track scoring architecture, on the one hand, relies on an accurate mathematical model to ensure the objectivity and consistency of the scoring; on the other hand, it uses a large model with a high degree of semantic parsing ability to enhance the subjective understanding and explanatory power of the scoring, greatly improving the accuracy and comprehensiveness of the evaluation. In addition, the above process is automated, so it is highly efficient, that is, the scoring and questioning method according to the answers to the interview questions of the present invention can quickly, accurately and comprehensively evaluate the applicant's answers. In addition, the scoring reasons and questioning questions given by the large model provide valuable information for digging out the applicant's knowledge mastery, alleviating the technical problem that the traditional question answer evaluation method cannot take into account the accuracy of the evaluation while ensuring efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of a method for scoring and questioning based on answers to interview questions provided by an embodiment of the present invention; Figure 2A schematic diagram of a device for scoring and questioning based on answers to interview questions provided by an embodiment of the present invention; Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Traditional question answer evaluation methods cannot ensure both efficiency and accuracy.

[0020] Based on this, the scoring and questioning method based on the answers to interview questions of the present invention integrates a variety of similarity measurement algorithms to achieve multi-angle quantitative evaluation of the applicant's answers, and also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that cannot be touched by pure mathematical calculations. This dual-track scoring architecture, on the one hand, relies on accurate mathematical models to ensure the objectivity and consistency of scoring; on the other hand, it uses a large model with a high degree of semantic parsing ability to enhance the subjective understanding and explanatory power of scoring, greatly improving the accuracy and comprehensiveness of the evaluation. In addition, the above process is automated, so it is highly efficient, that is, the scoring and questioning method based on the answers to interview questions of the present invention can quickly, accurately and comprehensively evaluate the applicant's answers. In addition, the scoring reasons and follow-up questions given by the large model provide valuable information for exploring the applicant's knowledge mastery.

[0021] To facilitate understanding of this embodiment, a method for scoring and questioning based on answers to interview questions disclosed in an embodiment of the present invention is first introduced in detail.

[0022] Embodiment 1: According to an embodiment of the present invention, an embodiment of a method for scoring and questioning based on answers to interview questions is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0023] Figure 1 is a flow chart of a method for scoring and questioning based on answers to interview questions according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps: Step S102: Obtain the candidate's answer to the current interview question, and perform vectorization processing on the candidate's answer and the preset reference answer to the current interview question to obtain the candidate's answer vector and the reference answer vector, where the reference answer includes at least one of the following: simple answer, standard answer, extended answer; In the embodiments of the present invention, five roles are involved, namely candidates, AI interview software (i.e., AI interview digital human), AI back-end service, AI large model, and embedding model. Specifically, the AI back-end service obtains the candidate's answer to the current interview question, and then inputs the candidate's answer and the preset reference answer to the current interview question into the embedding model for vectorization processing to obtain the candidate's answer vector and the reference answer vector.

[0024] The reference answers are divided into three levels: primary, intermediate, and advanced, corresponding to different weight scores respectively. The reference answer corresponding to the primary level is called a simple answer, the reference answer corresponding to the intermediate level is called a standard answer, and the reference answer corresponding to the advanced level is called an extended answer.

[0025] Different reference answers correspond to different depths of knowledge. Simple answer: The concept is simple and clear, easy to understand, focusing on intuitive understanding, without complex technical details; Standard answer: Pay attention to technical accuracy, avoid being too basic, be able to explain the core principles of the concept, and provide ideas for practical applications; Extended answer: Use a large number of technical terms and professional descriptions, focus on details and principles, describe accurately and completely, be able to relate multiple technical fields, and demonstrate the candidate's breadth and depth of knowledge.

[0026] Simple answer: Suitable for basic positions or non-technical scenarios, with the focus on enabling people to quickly understand the concept.

[0027] Standard answer: Suitable for intermediate positions or conventional technical evaluations, examining both basic knowledge and practical applications.

[0028] Extended answer: Suitable for senior positions, evaluating theoretical depth and technical professionalism, emphasizing details and innovative thinking.

[0029] For example Question: Please briefly describe the differences between machine learning and deep learning? Simple answer (primary answer): Machine learning is a method of using data to train models, including supervised learning, unsupervised learning, and reinforcement learning. Deep learning is a branch of machine learning that uses neural networks to automatically extract features and is suitable for processing complex data types such as images and speech. Weight 0.2.

[0030] Standard Answer (Intermediate Answer): Machine learning is a method of training models through data to complete specific tasks. It is divided into supervised learning, unsupervised learning, and reinforcement learning, and usually relies on manually designed feature engineering. Deep learning is a subfield of machine learning. Based on a multi-layer neural network architecture, it can automatically extract complex features from data, especially suitable for unstructured data (such as images, audio, and text), without manual feature extraction. Weight 0.3.

[0031] Extended Answer (Advanced Answer): Machine learning is a collection of algorithms that build predictive models by learning from labeled or unlabeled data. It relies on traditional statistical methods such as linear regression, support vector machines, and random forests, and usually requires human participation in feature engineering to optimize model performance. Deep learning is a subset of machine learning, and its core is a multi-layer neural network that can automatically learn feature representations through large-scale data and computing resources. The essence of deep learning lies in simulating the working mechanism of the human brain, which can extract hierarchical features of data in a high-dimensional space, thus achieving significant performance improvement in complex tasks (such as image recognition, natural language processing, etc.). However, its training process has high requirements for computing resources and data scale. Weight 0.5.

[0032] The above preset reference answers can be one of the simple answer, standard answer, and extended answer, or two of them, or even three of them, mainly depending on the administrator's customized configuration. If the reference answer is the simple answer, the obtained reference answer vector is the simple answer vector. If the reference answer is the simple answer and the standard answer, the obtained reference answer vector is the simple answer vector and the standard answer vector.

[0033] Step S104, calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the candidate's answer vector and the reference answer vector respectively; Specifically, the calculation processes of the above cosine similarity, Euclidean distance similarity, and Manhattan distance similarity are executed by the AI back-end service.

[0034] Step S106, perform a weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain the formula total score of the candidate's answer; Specifically, the process of calculating the above formula total score is executed by the AI back-end service.

[0035] Step S108, construct a scoring prompt word based on the current interview question, candidate's answer, reference answer, scoring rule, scoring standard, scoring reason, sampling temperature parameter, and maximum Token number, and input the scoring prompt word into the large model to output the large model total score and scoring reason of the candidate's answer; Specifically, the AI backend service constructs a scoring prompt word (such as the current interview question / candidate's answer / reference answer / scoring rule / invalid or avoidance criterion / scoring criterion / scoring reason / sampling temperature parameter / maximum token number) based on the current interview question, candidate's answer, reference answer, scoring rule from 0 to 100 (for example, a perfect answer gets 100 points, 80% match gets 80 points, 60% match gets 60 points, and so on), invalid or avoidance criteria (don't know, not clear, not familiar, not aware, not good at, not proficient in, can't, don't understand, haven't been exposed to, haven't covered, haven't mastered, haven't learned, skipped, etc.), scoring criteria, scoring reasons, sampling temperature parameters, maximum token number, etc., and then feeds it to the large model to output the total score of the candidate's answer by the large model and the reason for the score.

[0036] The AI large model conducts reasoning, evaluation, and scoring based on the scoring prompt word and gives the reason for the score. The AI backend service analyzes the result returned by the large model and extracts the score and the reason for the score.

[0037]

[0038] In step S110, a weighted average calculation is performed on the formula total score and the large model total score to obtain the final score of the candidate's answer, and the candidate's answer is evaluated based on the final score and the preset score threshold, and then subsequent processing is performed according to the evaluation result.

[0039] In an embodiment of the present invention, a scoring and questioning method based on answers to interview questions is provided, including: obtaining an answer of an applicant to a current interview question, and vectorizing the applicant's answer and a preset reference answer to the current interview question to obtain an applicant's answer vector and a reference answer vector, wherein the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; respectively calculating the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity of the applicant's answer vector and the reference answer vector; performing weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain a formula total score of the applicant's answer; constructing scoring prompt words based on the current interview question, the applicant's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of tokens, and inputting the scoring prompt words into a large model to output the large model total score and the scoring reason of the applicant's answer; performing weighted average calculation on the formula total score and the large model total score to obtain a final score of the applicant's answer, and evaluating the applicant's answer according to the final score and a preset score threshold, and then performing subsequent processing according to the evaluation result. From the above description, it can be seen that the scoring and questioning method according to the answers to interview questions of the present invention integrates a variety of similarity measurement algorithms to achieve multi-angle quantitative evaluation of the applicant's answers, and also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that cannot be touched by pure mathematical calculations. This dual-track scoring architecture, on the one hand, relies on an accurate mathematical model to ensure the objectivity and consistency of the scoring; on the other hand, it uses a large model with a high degree of semantic parsing ability to enhance the subjective understanding and explanatory power of the scoring, greatly improving the accuracy and comprehensiveness of the evaluation. In addition, the above process is automated, so it is highly efficient, that is, the scoring and questioning method according to the answers to interview questions of the present invention can quickly, accurately and comprehensively evaluate the applicant's answers. In addition, the scoring reasons and follow-up questions given by the large model provide valuable information for exploring the applicant's knowledge mastery, alleviating the technical problem that the traditional question answer evaluation method cannot take into account the accuracy of the evaluation while ensuring efficiency.

[0040] The above content briefly introduces the method for scoring and questioning based on answers to interview questions of the present invention. The specific contents involved are described in detail below.

[0041] In an optional embodiment of the present invention, obtaining the candidate's answer to the current interview question specifically includes the following steps: (1) When the applicant logs in to the AI ​​interview software, the AI ​​interview software obtains the corresponding target test questions from the test question bank based on the applicant's basic information, job position, and job level, and presents the current interview questions in the target test questions in text and online voice broadcast; When implemented, after the applicant logs in to the AI interview software, the AI interview digital human will retrieve target questions of corresponding difficulty from the question bank according to the applicant's basic information, the interviewed position, the required job level, etc., and then present the current interview question in the target questions in text and play it via online voice broadcast.

[0042] (2) Obtain the audio answer of the applicant to the current interview question, and convert the audio answer into text to obtain the applicant's answer.

[0043] Specifically, the applicant starts answering the current interview question. To improve the answering efficiency, the applicant will answer through the microphone.

[0044] For example, the applicant answers: "Machine learning is a method of using data to train models for solving specific tasks. It includes supervised learning, unsupervised learning, and reinforcement learning. Usually, it is necessary to manually design feature extraction methods to improve the model effect. Deep learning is a branch of machine learning that realizes the automatic extraction of complex features through a multi-layer neural network architecture, and is especially suitable for processing unstructured data without manual intervention in feature engineering." The AI interview digital human performs real-time recording and sends the audio answer to the AI back-end service. The AI back-end service converts the audio answer of the applicant's answer into text to obtain the applicant's answer.

[0045] In an optional embodiment of the present invention, the applicant's answer and the preset reference answer to the current interview question are vectorized, which specifically includes the following steps: Input the applicant's answer and the preset reference answer to the current interview question into an embedding model for vectorization to obtain the applicant's answer vector and the reference answer vector.

[0046] Specifically, the AI back-end service will retrieve the preset reference answer to the current interview question from the question bank, and perform vectorization (Embedding) with the applicant's answer input into the embedding model, and convert the reference answer and the applicant's answer into high-dimensional vectors A (reference answer vector) and B (applicant's answer vector).

[0047] The role of the embedding model (trained based on BERT) is to map high-dimensional sparse data (such as words or sentences) into a low-dimensional dense vector space to capture the semantic similarity between data. Here, BERT (Bidirectional Encoder Representations from Transformers), which performs excellently and is the most advanced in natural language processing tasks, is adopted. It is a bidirectional encoder based on the Transformer architecture. By introducing masked language modeling (MLM) and next sentence prediction (NSP), combined with the self-attention mechanism, it understands the context relationship of words in a sentence through unsupervised pre-training on a large amount of corpus, thereby capturing deeper semantic information in the text.

[0048] During pre-training, BERT randomly selects some tokens in the input text sequence for masking and lets the model predict these masked words. This allows the model to learn to infer the missing information based on the surrounding context. Specifically, BERT randomly selects 15% of the words in the input text sequence for masking. Then, 80% of them are replaced with a special [MASK] token, 10% remain unchanged, and the other 10% are randomly replaced with other words. That is, assuming we have an input sequence containing 100 words, according to BERT's masking strategy, 15% of the words will be selected for masking: 15% of 100 words is 15 words, which will be selected for masking. Among these 15 selected words: 80% of the words (i.e., 12 out of 15 words) will be replaced with the [MASK] token. 10% of the words (i.e., 1 out of 15 words) will remain unchanged. 10% of the words (i.e., 2 out of 15 words) will be randomly replaced with other words. This is done to enable the model to learn not only the context information of the masked words but also the context information of the unmasked words, thereby improving the model's generalization ability.

[0049] The vectors generated by Embedding have multiple dimensions, such as 512 dimensions, 768 dimensions, 1024 dimensions. Each dimension is a floating-point number (float), which is used to represent the mapping value of words in the high-dimensional space, reflecting the relative position and semantic relationship characteristics of words in the multi-dimensional space. The higher the precision of the floating-point value, the more subtle differences and complex relationships in semantics can be captured.

[0050] Dimensions greater than three cannot be visually represented by a graph. However, whether it is high-dimensional or low-dimensional, their calculation methods are the same. Therefore, next, a visual three-dimensional vector will be used as an example for calculation and explanation.

[0051] Example: Assume that the standard answer in the reference answer is represented as a vector after vectorization (i.e., the standard answer vector):

[0052] The applicant's answer vector is represented as:

[0053] In an alternative embodiment of the present invention, the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the applicant's answer vector and the reference answer vector are calculated respectively, which specifically include the following steps: (1) Calculate the cosine similarity between the applicant's answer vector and the reference answer vector according to the cosine similarity calculation formula where, represents the cosine similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the applicant's answer vector, represents the dimension of the applicant's answer vector or the reference answer vector; (2) Calculate the Euclidean distance similarity between the applicant's answer vector and the reference answer vector according to the Euclidean distance similarity calculation formula where, represents the Euclidean distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the applicant's answer vector, represents the dimension of the applicant's answer vector or the reference answer vector; (3) Calculate the Manhattan distance similarity between the applicant's answer vector and the reference answer vector according to the Manhattan distance similarity calculation formula where, represents the Manhattan distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the applicant's answer vector, represents the dimension of the applicant's answer vector or the reference answer vector.

[0054] Specifically, the AI back-end service calculates the cosine similarity between the applicant's answer vector and the reference answer vector: Calculate: ;

[0055]

[0056] Calculate: ;

[0057] Calculate: ;

[0058]

[0059] The AI back-end service calculates the Euclidean distance similarity between the candidate's answer vector and the reference answer vector: Calculate item by item:

[0060] Accumulate and take the square root to make the result consistent with the original scale:

[0061]

[0062] The AI back-end service calculates the Manhattan distance similarity between the candidate's answer vector and the reference answer vector: Calculate item by item:

[0063] Accumulate the scores:

[0064]

[0065] If the similarity is 1, it means that the candidate's answer is exactly the same as the standard answer; if it is 0, it means that the candidate's answer is completely irrelevant to the standard answer. The closer it is to 1, the more similar it is, and the closer it is to 0, the less similar it is.

[0066] In an alternative embodiment of the present invention, if the cosine similarity includes: the cosine similarity between the candidate's answer and the simple answer, the cosine similarity between the candidate's answer and the standard answer, the cosine similarity between the candidate's answer and the extended answer, the Euclidean distance similarity includes: the Euclidean distance similarity between the candidate's answer and the simple answer, the Euclidean distance similarity between the candidate's answer and the standard answer, the Euclidean distance similarity between the candidate's answer and the extended answer, and the Manhattan distance similarity includes: the Manhattan distance similarity between the candidate's answer and the simple answer, the Manhattan distance similarity between the candidate's answer and the standard answer, the Manhattan distance similarity between the candidate's answer and the extended answer, a weighted average calculation is performed on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity, specifically including the following steps: According to the formula for calculating the total score 、 、 、 Calculate the total score of the candidate's answer formula, where represents the total score of the candidate's answer formula, represents the total score of the cosine similarity, represents the total score of the Euclidean distance similarity, Represents the total Manhattan distance similarity score, Represents the weight corresponding to the total cosine similarity score, Represents the weight corresponding to the total Euclidean distance similarity score, Represents the weight corresponding to the total Manhattan distance similarity score, Represents the cosine similarity between the candidate's answer and the simple answer, Represents the cosine similarity between the candidate's answer and the standard answer, Represents the cosine similarity between the candidate's answer and the extended answer, Represents the Euclidean distance similarity between the candidate's answer and the simple answer, Represents the Euclidean distance similarity between the candidate's answer and the standard answer, Represents the Euclidean distance similarity between the candidate's answer and the extended answer, Represents the Manhattan distance similarity between the candidate's answer and the simple answer, Represents the Manhattan distance similarity between the candidate's answer and the standard answer, Represents the Manhattan distance similarity between the candidate's answer and the extended answer, Represents the weight corresponding to the simple answer, Represents the weight corresponding to the standard answer, Represents the weight corresponding to the extended answer.

[0067] Specifically, for the AI back-end service, the weighted average method is used for the calculation formula of the total score. The formula is: Total score = (Cosine similarity total score × 0.6 weight) + (Euclidean distance similarity total score × 0.2 weight) + (Manhattan distance similarity total score × 0.2 weight), and then it is converted into a score between 0 and 100. Among them, the weight value can be customized according to the actual scenario.

[0068] As the above example: If the cosine similarity is the cosine similarity between the candidate's answer and the standard answer, then the total cosine similarity score = the cosine similarity between the candidate's answer and the standard answer (0.9745). If the Euclidean distance similarity is the Euclidean distance similarity between the candidate's answer and the standard answer, then the total Euclidean distance similarity score = the Euclidean distance similarity between the candidate's answer and the standard answer (0.76). If the Manhattan distance similarity is the Manhattan distance similarity between the candidate's answer and the standard answer, then the total Manhattan distance similarity score = the Manhattan distance similarity between the candidate's answer and the standard answer (0.69).

[0069] Total score =

[0070] The above calculation process is only the calculation process of the standard answer and the candidate's answer. In the actual application process, parallel tensor matrix calculations will also be performed with the simple answer (primary answer) and the extended answer (advanced answer). The calculation process is the same, but the data is different. Whether to calculate depends on the configuration of the interviewer.

[0071] The reason for this design is that intermediate candidates may also answer more difficult answers, so they should receive a higher weighted score.

[0072] Therefore, the formula for calculating the total score is as follows: 、 、 、 , where represents the total formula score of the candidate's answer, represents the total cosine similarity score, represents the total Euclidean distance similarity score, represents the total Manhattan distance similarity score, represents the weight corresponding to the total cosine similarity score, represents the weight corresponding to the total Euclidean distance similarity score, represents the weight corresponding to the total Manhattan distance similarity score, represents the cosine similarity between the candidate's answer and the simple answer, represents the cosine similarity between the candidate's answer and the standard answer, represents the cosine similarity between the candidate's answer and the extended answer, represents the Euclidean distance similarity between the candidate's answer and the simple answer, represents the Euclidean distance similarity between the candidate's answer and the standard answer, represents the Euclidean distance similarity between the candidate's answer and the extended answer, represents the Manhattan distance similarity between the candidate's answer and the simple answer, represents the Manhattan distance similarity between the candidate's answer and the standard answer, represents the Manhattan distance similarity between the candidate's answer and the extended answer, represents the weight corresponding to the simple answer, represents the weight corresponding to the standard answer, represents the weight corresponding to the extended answer.

[0073] In an alternative embodiment of the present invention, a weighted average calculation is performed on the formula total score and the large model total score, specifically including the following steps: According to the final score calculation formula Calculate the final score of the candidate's answer, where represents the final score of the candidate's answer, Represents the total formula score, Represents the total score of the large model, Represents the weight corresponding to the total formula score, Represents the weight corresponding to the total score of the large model.

[0074] Specifically, the AI backend service recalculates the final score of the candidate's answer. The weighted average formula is: Final score = (Explicit algorithm logic score (i.e., total formula score) * 0.6 weight) + (Large model score (i.e., total score of the large model) * 0.4 weight), where the weight value can be configured as needed.

[0075] As the above example, =

[0076] Converted to a percentage, and finally the final score is 82.

[0077] By combining explicit algorithms such as cosine similarity, Euclidean distance similarity, and Manhattan distance similarity with the large model score, the final score is calculated, and the final score is 82 points.

[0078] The explicit algorithm quantifies the matching degree between the candidate's answer and the standard answer through mathematical formulas, providing interpretability and transparency; while the large model score uses the context understanding ability of deep learning to comprehensively evaluate semantics, logic, and language quality, making up for the deficiency of the explicit algorithm in understanding complex semantic relationships and further supplementing the explicit algorithm. The combination of these two methods not only ensures the scientificity, objectivity, and fairness of the scoring process but also enhances the flexibility and semantic adaptation ability of the evaluation, achieving a balance between high precision and high interpretability.

[0079] In an optional embodiment of the present invention, the candidate's answer is evaluated according to the final score and the preset score threshold, and then subsequent processing is performed according to the evaluation result, which specifically includes the following steps: (1) If the final score is not less than the preset score threshold, it is determined that the candidate's answer is complete and valid, and proceed to the next interview question; (2) If the final score is less than the preset score threshold and the number of follow-up questions has not been exceeded, a prompt word for generating a follow-up question is constructed based on the current interview question, the candidate's answer, the reference answer, the sampling temperature parameter, and the description of the content lacking in the candidate's answer, and the prompt word for generating a follow-up question is input into the large model to output the follow-up question, and then the candidate is asked to answer the follow-up question. The follow-up question is generated for the content lacking in the candidate's answer; (3) If the final score is less than the preset score threshold, the number of follow-up questions has been exceeded, or the candidate's answer is determined to be an invalid answer, proceed to the next interview question.

[0080] Specifically, the AI back-end service determines whether the final score is not less than the specified preset score threshold. If the final score is not less than the preset score threshold, it is determined that the answer is complete and valid, and the next interview question is entered; if the final score is less than the preset score threshold, it is determined that the answer content is lacking. Under this condition, it is continued to determine whether the current interview question has reached the maximum number of follow-up questions. If the maximum number of follow-up questions has not been reached, a follow-up question prompt word will be reconstructed and generated again (the applicant's answer, the current interview question, the reference answer, the sampling temperature parameter, the description of generating the follow-up question: Please compare the content of the user's answer with the standard answer / the answer generated by the large model according to the current interview question to evaluate the deficiencies in the content of the user's answer, and generate new follow-up questions based on the deficiencies). The large model will regenerate the follow-up question for the lacking content, then convert the follow-up question from text to speech, and finally the AI interview digital human will perform voice broadcast to the applicant, so that the applicant can answer the follow-up question, and the above processes from step S102 to step S110 will be carried out again to evaluate the follow-up question. If the final score is less than the preset score threshold and the maximum number of follow-up questions for the current interview question is reached, the next interview question will continue to be broadcast. Another situation is that the applicant's score is 0, the answer is wrong, the answer is invalid, or the answer is refused. After reaching the maximum number of follow-up questions for the current interview question, this current interview question will also be skipped and the next interview question will be broadcast. The above all processes of evaluating each interview question in a loop are carried out until the applicant answers all the interview questions in the target test questions, and finally it ends.

[0081] In the method of the present invention, first, the applicant logs in to the AI interview software and answers the interview questions. Then, the applicant's answer and the reference answers of three difficulty levels of simple, standard, and advanced set in advance are vectorized, and the corresponding score values are calculated through the Euclidean distance similarity, Manhattan distance similarity, and cosine similarity. Each similarity algorithm is assigned different weights according to its characteristics, and each reference answer level is also assigned different weights to ensure that the scoring result can comprehensively consider the applicant's answer in all aspects. Then, the explicit formula total score is calculated by using the multi-layer weighted average method.

[0082] Next, the interview question, the reference answers of three difficulty levels, and the applicant's answer, etc. are constructed into a scoring prompt word and input into the pre-trained large model together to obtain a more intelligent large model total score. Then, the explicit formula total score calculated and the large model total score are weighted and synthesized in proportion to obtain the final score. If the final score reaches the preset score threshold, it is considered that the answer is valid and the next interview question can be entered; otherwise, if the final score is lower than the preset score threshold and the number of follow-up questions has not been exceeded, the applicant is allowed to further clarify the answer content of the follow-up question; and when the final score is lower than the preset score threshold and the number of follow-up questions has been exceeded, or the answer is determined to be invalid, there will be no further follow-up questions, and the next interview question will be directly entered.

[0083] Different from the scoring method purely based on mathematical formulas, the large model scoring not only provides the total score, but also gives detailed scoring reasons, points out the advantages and disadvantages in the candidate's answer, and generates follow-up questions for incomplete or lacking parts to understand the candidate's knowledge level more deeply.

[0084] Adopting this dual-track scoring mechanism not only achieves multi-dimensional accurate evaluation, but also enhances the transparency and rationality of scoring, making the entire evaluation process more scientific and fair.

[0085] The purpose of the present invention is to develop an efficient, fair and intelligent automatic scoring and evaluation mechanism, which aims to meet the need for rapid and accurate evaluation of candidates' answers to questions.

[0086] This solution has innovation in many aspects: First, it combines multiple similarity algorithm scores. Instead of using a single algorithm to measure the similarity of answers, by assigning different algorithm weights, it can not only reflect the absolute difference but also the semantic relevance, comprehensively considering the similarity degree of answers in different dimensions, making the scoring result more comprehensive and scientific. Euclidean distance focuses on the actual distance between two points in the vector space, Manhattan distance pays more attention to the sum of distances along the coordinate axes, and cosine similarity focuses on the consistency of vector directions. By comprehensively using these three algorithms and making full use of the advantages of each algorithm, the relationship between answers can be analyzed from multiple angles.

[0087] Second, it has a hierarchical weighting mechanism for answers of different difficulty levels. Simple answers, standard answers, and extended answers contribute differently to evaluating the quality of candidates' answers. By reasonably allocating weights, the evaluation can more accurately reflect the actual level of candidates' answers. Simple answers may cover the basic points, and their higher weights can ensure key consideration of the mastery of basic content; while advanced answers reflect a deeper understanding.

[0088] Third, it introduces a large model based on deep learning for scoring. The large model has powerful context language understanding and analysis capabilities, can evaluate answers from multiple complex levels such as semantics, logic, and integrity, making up for the deficiencies of relying solely on similarity algorithms in semantic understanding and logical coherence analysis. The scoring reasons and follow-up questions given by the large model provide valuable information for further exploring the candidates' knowledge mastery.

[0089] In summary, the main purpose of the present invention is to establish an efficient, fair, scientific and humanized scoring and questioning mechanism, which provides strong support and reliable guarantee for remote interviews of enterprises, greatly reduces labor costs and improves interview efficiency. The innovation lies in that it breaks through the limitations of traditional scoring methods and establishes a flexible, intelligent and efficient scoring framework by integrating a variety of advanced technical means, bringing new changes to the field of automated evaluation.

[0090] The key points of the present invention are: The key protection point is to build a unique dual-track hybrid scoring system, which not only integrates multiple distance measurement algorithms to achieve multi-angle quantitative evaluation of applicants' answers, but also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that pure mathematical calculations cannot reach. This dual-track scoring architecture relies on precise mathematical models to ensure the objectivity and consistency of scoring on the one hand; on the other hand, it uses a large model with high semantic parsing capabilities to enhance the subjective understanding and explanatory power of scoring. In order to maintain the transparency of scoring, the large model will also provide specific reasons for each scoring (i.e., scoring reasons) so that applicants can clearly understand their strengths and weaknesses. At the same time, considering the value differences of different types of reference answers, the present invention specially designs a hierarchical weighting mechanism, that is, different weights are assigned and customized according to the difficulty of the reference answers to ensure that the scoring results can objectively and accurately reflect the actual level of the answerer. In addition, the present invention emphasizes the rebalancing of the output results of different similarity algorithms, and optimizes the composition of the final score by setting their respective weights, avoiding the possibility of a single algorithm dominating the scoring process.

[0091] The present invention also pays special attention to the interactivity and dynamic adjustment capabilities in the scoring process. By feeding the interview questions, simple answers, standard answers, advanced answers, and prompt words constructed by the applicant's answers to the big model, not only can an overall score be obtained, but also potential problems in the applicant's answers can be identified. For those incomplete and incomplete places, the big model will automatically generate follow-up questions to further verify the applicant's depth of understanding and mastery of knowledge. This mechanism not only increases the flexibility of scoring, but also provides applicants with more opportunities to demonstrate their abilities.

[0092] In general, the key protection points of the present invention are reflected in the following aspects: first, a multi-dimensional quantitative evaluation model is established, which ensures the comprehensiveness and accuracy of the scoring results by combining different distance measurement algorithms and hierarchical weighting mechanisms; second, a large model with powerful semantic parsing capabilities is introduced to improve the subjective understanding and explanatory power of the scoring; third, the transparency and interactivity of the scoring process are emphasized, and the applicant experience is enhanced by providing detailed scoring reasons and generating follow-up questions; fourth, reasonable preset score thresholds and follow-up question times are set to ensure the scientificity and fairness of the evaluation results.

[0093] Embodiment 2: The embodiment of the present invention also provides a scoring and follow-up device based on the answers to interview questions. This scoring and follow-up device based on the answers to interview questions is mainly used to execute the scoring and follow-up method based on the answers to interview questions provided in Embodiment 1 of the present invention. The following is a specific introduction to the scoring and follow-up device based on the answers to interview questions provided in the embodiment of the present invention.

[0094] Figure 2 It is a schematic diagram of a scoring and follow-up device based on the answers to interview questions according to an embodiment of the present invention. As Figure 2 shown, the device mainly includes: a vectorization processing unit 10, a calculation unit 20, a weighted average calculation unit 30, a scoring prompt word construction unit 40, and a weighted average calculation and evaluation unit 50, where: The vectorization processing unit is used to obtain the applicant's answer to the current interview question, and perform vectorization processing on the applicant's answer and the preset reference answer to the current interview question to obtain the applicant's answer vector and the reference answer vector. Among them, the reference answer includes at least one of the following: simple answer, standard answer, and extended answer; The calculation unit is used to calculate the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity between the applicant's answer vector and the reference answer vector respectively; The weighted average calculation unit is used to perform weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain the formula total score of the applicant's answer; The scoring prompt word construction unit is used to construct a scoring prompt word based on the current interview question, the applicant's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of Tokens, and input the scoring prompt word into the large model to output the large model total score and the scoring reason of the applicant's answer; The weighted average calculation and evaluation unit is used to perform weighted average calculation on the formula total score and the large model total score to obtain the final score of the applicant's answer, and evaluate the applicant's answer according to the final score and the preset score threshold, and then perform subsequent processing according to the evaluation result.

[0095] In an embodiment of the present invention, a scoring and questioning device based on answers to interview questions is provided, comprising: obtaining an answer of an applicant to a current interview question, and vectorizing the applicant's answer and a preset reference answer to the current interview question to obtain an applicant's answer vector and a reference answer vector, wherein the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; respectively calculating the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity of the applicant's answer vector and the reference answer vector; performing weighted average calculation on the cosine similarity, Euclidean distance similarity, and Manhattan distance similarity to obtain a formula total score of the applicant's answer; constructing scoring prompt words based on the current interview question, the applicant's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of tokens, and inputting the scoring prompt words into a large model to output the large model total score and the scoring reason of the applicant's answer; performing weighted average calculation on the formula total score and the large model total score to obtain a final score of the applicant's answer, and evaluating the applicant's answer according to the final score and a preset score threshold, and then performing subsequent processing according to the evaluation result. From the above description, it can be known that the scoring and questioning device according to the answers to the interview questions of the present invention integrates a variety of similarity measurement algorithms to achieve a multi-angle quantitative evaluation of the applicant's answers, and also creatively adds a large model scoring link based on deep learning to make up for the depth of language understanding that cannot be touched by pure mathematical calculations. This dual-track scoring architecture, on the one hand, relies on an accurate mathematical model to ensure the objectivity and consistency of the scoring; on the other hand, it uses a large model with a high degree of semantic parsing ability to enhance the subjective understanding and explanatory power of the scoring, greatly improving the accuracy and comprehensiveness of the evaluation. In addition, the above process is automated, so it is highly efficient, that is, the scoring and questioning method according to the answers to the interview questions of the present invention can quickly, accurately and comprehensively evaluate the applicant's answers. In addition, the scoring reasons and follow-up questions given by the large model provide valuable information for exploring the applicant's knowledge mastery, alleviating the technical problem that the traditional question answer evaluation method cannot take into account the accuracy of the evaluation while ensuring efficiency.

[0096] Optionally, the vectorized processing unit is also used for: when the applicant logs into the AI ​​interview software, the AI ​​interview software obtains the corresponding target questions from the question bank based on the applicant's basic information, applied position, and position level, and presents the current interview questions in the target questions in text and online voice broadcast; obtains the audio answers of the applicant to the current interview questions, and converts the audio answers into text to obtain the applicant's answers.

[0097] Optionally, the vectorization processing unit is also used to: input the applicant's answer and a preset reference answer to the current interview question into the embedding model for vectorization processing to obtain an applicant's answer vector and a reference answer vector.

[0098] Optionally, the calculation unit is further configured to: according to the cosine similarity calculation formula calculate the cosine similarity between the applicant's answer vector and the reference answer vector, where represents the cosine similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the applicant's answer vector, represents the dimension of the applicant's answer vector or the reference answer vector; according to the Euclidean distance similarity calculation formula calculate the Euclidean distance similarity between the applicant's answer vector and the reference answer vector, where represents the Euclidean distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the applicant's answer vector, represents the dimension of the applicant's answer vector or the reference answer vector; according to the Manhattan distance similarity calculation formula calculate the Manhattan distance similarity between the applicant's answer vector and the reference answer vector, where represents the Manhattan distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the applicant's answer vector, represents the dimension of the applicant's answer vector or the reference answer vector.

[0099] Optionally, if the cosine similarity includes: the cosine similarity between the applicant's answer and the simple answer, the cosine similarity between the applicant's answer and the standard answer, the cosine similarity between the applicant's answer and the extended answer, the Euclidean distance similarity includes: the Euclidean distance similarity between the applicant's answer and the simple answer, the Euclidean distance similarity between the applicant's answer and the standard answer, the Euclidean distance similarity between the applicant's answer and the extended answer, and the Manhattan distance similarity includes: the Manhattan distance similarity between the applicant's answer and the simple answer, the Manhattan distance similarity between the applicant's answer and the standard answer, the Manhattan distance similarity between the applicant's answer and the extended answer, the weighted average calculation unit is further configured to: according to the formula total score calculation formula , , , calculate the formula total score of the applicant's answer, where represents the formula total score of the applicant's answer, represents the total score of the cosine similarity, represents the total score of the Euclidean distance similarity, represents the total score of the Manhattan distance similarity, represents the weight corresponding to the total score of the cosine similarity, represents the weight corresponding to the total score of the Euclidean distance similarity, Represents the weight corresponding to the total Manhattan distance similarity score, Represents the cosine similarity between the candidate's answer and the simple answer, Represents the cosine similarity between the candidate's answer and the standard answer, Represents the cosine similarity between the candidate's answer and the extended answer, Represents the Euclidean distance similarity between the candidate's answer and the simple answer, Represents the Euclidean distance similarity between the candidate's answer and the standard answer, Represents the Euclidean distance similarity between the candidate's answer and the extended answer, Represents the Manhattan distance similarity between the candidate's answer and the simple answer, Represents the Manhattan distance similarity between the candidate's answer and the standard answer, Represents the Manhattan distance similarity between the candidate's answer and the extended answer, Represents the weight corresponding to the simple answer, Represents the weight corresponding to the standard answer, Represents the weight corresponding to the extended answer.

[0100] Optionally, the weighted average calculation and evaluation unit is also used to: calculate the final score of the candidate's answer according to the final score calculation formula where, Represents the final score of the candidate's answer, Represents the total score of the formula, Represents the total score of the large model, Represents the weight corresponding to the total score of the formula, Represents the weight corresponding to the total score of the large model.

[0101] Optionally, the weighted average calculation and evaluation unit is also used to: if the final score is not less than the preset score threshold, determine that the candidate's answer is complete and valid, and proceed to the next interview question; if the final score is less than the preset score threshold and the number of follow-up questions has not been exceeded, then based on the current interview question, the candidate's answer, the reference answer, the sampling temperature parameter, construct a prompt for generating a follow-up question to describe the generated follow-up question, and input the generated follow-up question prompt into the large model to output the follow-up question, so that the candidate answers the follow-up question, where the follow-up question is generated for the content lacking in the candidate's answer; if the final score is less than the preset score threshold and the number of follow-up questions has been exceeded, or the candidate's answer is determined to be an invalid answer, then proceed to the next interview question.

[0102] The device provided by the embodiments of the present invention has the same implementation principle and the same technical effects as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding contents in the foregoing method embodiments.

[0103] Such asFigure 3 As shown in Figure 3 , an electronic device 600 provided by an embodiment of the present application includes: a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device runs, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the method for scoring and follow-up questioning according to the interview question answers as described above.

[0104] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memory and processor, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, it can execute the method for scoring and follow-up questioning according to the interview question answers as described above.

[0105] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 601 or instructions in software form. The above-mentioned processor 601 can be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it can also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and combines its hardware to complete the steps of the above method.

[0106] The scoring follow-up device according to the interview question answers provided by the embodiments of the present application can be specific hardware on the device, or software or firmware installed on the device, etc. For the device provided by the embodiments of the present application, its implementation principle and the resulting technical effects are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the foregoing described systems, devices, and units can all refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0107] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0108] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the device, method, and computer program product according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of the code, and the module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of the blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0109] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, each functional unit in the embodiments provided in this application may be integrated into one processing unit, may exist physically alone for each unit, or two or more units may be integrated into one unit.

[0111] If the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method for scoring and follow-up questioning according to the interview question answers described in each embodiment of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0112] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0113] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of this application, used to illustrate the technical solution of this application, rather than limiting it. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed in this application can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for scoring and questioning based on answers to interview questions, characterized in that: include: Obtaining the candidate's answer to the current interview question, and vectorizing the candidate's answer and a preset reference answer to the current interview question to obtain a candidate's answer vector and a reference answer vector, wherein the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; Calculating the cosine similarity, Euclidean distance similarity and Manhattan distance similarity of the applicant's answer vector and the reference answer vector respectively; Performing weighted average calculation on the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity to obtain a formula total score of the applicant's answer; Based on the current interview question, the candidate's answer, the reference answer, the scoring rules, the scoring criteria, the scoring reasons, the sampling temperature parameter, and the maximum number of tokens, a scoring prompt is constructed, and the scoring prompt is input into the big model to output the big model total score and the scoring reason of the candidate's answer; The total score of the formula and the total score of the large model are weighted averaged to obtain a final score of the applicant's answer, and the applicant's answer is evaluated based on the final score and a preset score threshold, and subsequent processing is performed based on the evaluation result.

2. The method according to claim 1, characterized in that Get candidate answers to current interview questions, including: When the applicant logs in to the AI ​​interview software, the AI ​​interview software obtains the corresponding target test questions from the test question bank according to the applicant's basic information, job position, and job level, and presents the current interview questions in the target test questions in text and online voice broadcast; The audio answer of the candidate to the current interview question is obtained, and the audio answer is converted into text to obtain the candidate's answer.

3. The method according to claim 1, characterized in that Vectorizing the applicant's answer and the preset reference answer to the current interview question, including: The candidate's answer and the preset reference answer to the current interview question are input into the embedding model for vectorization processing to obtain the candidate's answer vector and the reference answer vector.

4. The method according to claim 1, characterized in that Calculating the cosine similarity, Euclidean distance similarity and Manhattan distance similarity of the applicant answer vector and the reference answer vector respectively, including: According to the cosine similarity calculation formula Calculate the cosine similarity between the candidate's answer vector and the reference answer vector, where: represents the cosine similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, Represents the dimension of the candidate's answer vector or the reference answer vector; According to the Euclidean distance similarity calculation formula Calculate the Euclidean distance similarity between the candidate's answer vector and the reference answer vector, where: represents the Euclidean distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, Represents the dimension of the candidate's answer vector or the reference answer vector; Based on the Manhattan distance similarity calculation formula Calculate the Manhattan distance similarity between the candidate's answer vector and the reference answer vector, where: represents the Manhattan distance similarity, represents the i-th dimension of the reference answer vector, represents the i-th dimension of the candidate's answer vector, Represents the dimension of the applicant's answer vector or the reference answer vector.

5. The method according to claim 1, characterized in that If the cosine similarity includes: the cosine similarity between the candidate's answer and the simple answer, the cosine similarity between the candidate's answer and the standard answer, and the cosine similarity between the candidate's answer and the extended answer, the Euclidean distance similarity includes: the Euclidean distance similarity between the candidate's answer and the simple answer, the Euclidean distance similarity between the candidate's answer and the standard answer, and the Euclidean distance similarity between the candidate's answer and the extended answer, and the Manhattan distance similarity includes: the Manhattan distance similarity between the candidate's answer and the simple answer, the Manhattan distance similarity between the candidate's answer and the standard answer, and the Manhattan distance similarity between the candidate's answer and the extended answer, weighted average calculation of the cosine similarity, the Euclidean distance similarity, and the Manhattan distance similarity includes: Calculate the total score according to the formula , , , Calculate the total score of the candidate's answer, where a formula totaling the candidate's answers, represents the total cosine similarity score, represents the total score of Euclidean distance similarity, represents the total Manhattan distance similarity score, Represents the weight corresponding to the total score of cosine similarity, Represents the weight corresponding to the total score of Euclidean distance similarity, Represents the weight corresponding to the total score of Manhattan distance similarity, represents the cosine similarity between the candidate's answer and the simple answer, represents the cosine similarity between the applicant's answer and the standard answer, represents the cosine similarity between the applicant's answer and the extended answer, represents the Euclidean distance similarity between the candidate's answer and the simple answer, represents the Euclidean distance similarity between the applicant's answer and the standard answer, represents the Euclidean distance similarity between the candidate's answer and the extended answer, represents the Manhattan distance similarity between the candidate's answer and the simple answer, represents the Manhattan distance similarity between the candidate's answer and the standard answer, represents the Manhattan distance similarity between the candidate's answer and the extended answer, represents the weight corresponding to the simple answer, represents the weight corresponding to the standard answer, Indicates the weight corresponding to the extended answer.

6. The method according to claim 1, characterized in that The weighted average calculation of the total score of the formula and the total score of the large model includes: Calculate the formula based on the final score Calculate the final score of the candidate's answer, where: represents the final score of the candidate's answer, represents the total score of the formula, represents the total score of the large model, represents the weight corresponding to the total score of the formula, Indicates the weight corresponding to the total score of the large model.

7. The method according to claim 1, characterized in that The applicant's answer is evaluated according to the final score and the preset score threshold, and subsequent processing is performed according to the evaluation result, including: If the final score is not less than the preset score threshold, the applicant's answer is determined to be complete and valid, and the next interview question is entered; If the final score is less than the preset score threshold and does not exceed the number of follow-up questions, a prompt word for generating a follow-up question is constructed based on the current interview question, the candidate's answer, the reference answer, the sampling temperature parameter, and the description of generating the follow-up question, and the prompt word for generating the follow-up question is input into the large model, and a follow-up question is output, so that the candidate can answer the follow-up question, wherein the follow-up question is generated for the content that is missing from the candidate's answer; If the final score is less than the preset score threshold and exceeds the number of follow-up questions, or the applicant's answer is determined to be an invalid answer, the next interview question will be entered.

8. A device for scoring and questioning based on answers to interview questions, characterized in that: include: A vectorization processing unit is used to obtain the candidate's answer to the current interview question, and vectorize the candidate's answer and the preset reference answer of the current interview question to obtain the candidate's answer vector and the reference answer vector, wherein the reference answer includes at least one of the following: a simple answer, a standard answer, and an extended answer; A calculation unit, used to calculate the cosine similarity, Euclidean distance similarity and Manhattan distance similarity of the applicant's answer vector and the reference answer vector respectively; A weighted average calculation unit, used for performing weighted average calculation on the cosine similarity, the Euclidean distance similarity and the Manhattan distance similarity to obtain a total formula score of the applicant's answer; A scoring prompt word construction unit is used to construct a scoring prompt word based on the current interview question, the applicant's answer, the reference answer, the scoring rule, the scoring standard, the scoring reason, the sampling temperature parameter, and the maximum number of tokens, and input the scoring prompt word into the big model to output the big model total score and the scoring reason of the applicant's answer; The weighted average calculation and evaluation unit is used to perform weighted average calculation on the total score of the formula and the total score of the large model to obtain the final score of the applicant's answer, and evaluate the applicant's answer based on the final score and a preset score threshold, and then perform subsequent processing based on the evaluation result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images